Transparent Proxy Architecture for Multi-Path Data Connections

ABSTRACT

A method for forming an optimized communication connection providing enhanced performance of an application utilizing the communication connection is provided, the communication connection including multiple individual communication networks. The method includes: obtaining a set of performance requirements corresponding to the application utilizing the communication connection; obtaining real-time capacity information for each of a plurality of available channels associated with the respective individual communication networks; applying at least one policy-based management criteria to the available channels for controlling, in real-time, one or more aspects of the available channels; dynamically aggregating the individual communication networks to form the optimized communication connection, the communication connection leveraging one or more features and capabilities of at least a subset of the communication networks; and controlling real-time traffic scheduling across at least a subset of the available channels so as to adapt the communication connection to changes in network conditions and/or policy-based management criteria.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of pending U.S. application Ser. No.13/807,740 filed on Dec. 30, 2012, the disclosure of which isincorporated herein by reference in its entirety. U.S. application Ser.No. 13/807,740 is a national stage entry under 35 U.S.C. §371 of PatentCooperation Treaty (PCT) Application No. PCT/US11/43461 filed on Jul. 8,2011, which in turn claims the benefit of U.S. Provisional ApplicationNo. 61/362,764 filed on Jul. 9, 2010, the disclosures of which areincorporated herein by reference in their entirety.

FIELD OF THE INVENTION

The present invention relates generally to the electrical, electronic,and computer arts, and more particularly relates to electronic devicecommunications.

BACKGROUND OF THE INVENTION

The Internet provides a platform for rapid and timely informationexchange among a disparate array of clients and servers. Transmissioncontrol protocol (TCP) and Internet protocol (IP) are well-knownseparately designed and closely tied protocols that define rules ofcommunication between end hosts, and are presently the most commonlyused protocol suite for transferring data via the Internet. Thecombination of TCP/IP dominates modern communications over variousnetworks, from a wired backbone to a heterogeneous network, dueprimarily to its simplicity and reliability. TCP has become the de factostandard used in most applications ranging from interactive sessions,such as, for example, hypertext transfer protocol (HTTP), to bulk datatransfer, such as, for example, file transfer protocol (FTP). UserDatagram Protocol (UDP) is used by many other applications, particularlyInternet applications, requiring timely delivery of data. However, dueat least in part to the lack of implicit acknowledgments, UDP cannotguarantee reliability, ordering or integrity of the data. Mostmultimedia streaming applications, as well as Voice over IP (VoIP), usethe UDP protocol.

TCP was originally designed primarily for wired networks. In a wirednetwork, random bit error rate (BER), a characteristic usually morepronounced in a wireless network environment, is negligible, andcongestion is a main cause of packet loss. Emerging wirelessapplications, especially high-speed multimedia services and the adventof wireless IP communications carried by the Internet, call forcalibration and sophisticated enhancement or modifications of thisprotocol suite for improved performance.

Based on the assumption that packet losses are indicative of networkcongestion, an additive increase/multiplicative decrease (AIMD)algorithm, which is a feedback control mechanism used in TCP congestionavoidance, reaches a steady state which reflects the protocol'sefficiency in terms of throughput and link utilization. However, thisassumption does not necessarily hold true when the end-to-end path alsoincludes wireless links. Factors including, for example, high BER,unstable channel characteristics, and user mobility, may all contributeto packet losses.

Many studies have shown that an unmodified standard TCP performs poorlyin a wireless communications environment due, at least in part, to itsinability to distinguish packet losses caused by network congestion fromthose losses attributed to transmission errors. This phenomenon getsexacerbated for very lossy wireless connections such as, for example,satellite. Numerous modifications to TCP have been proposed to resolve,or at least alleviate, some of the issues with its original design.Unfortunately, however, conventional methodologies for addressing theseissues have been, thus far, ineffective.

To be effective, modifications to TCP should to be applied on both endsof the communications (e.g., on the client side and on the server side).In practice, however, it is very rare to control both endpoints. Assuch, the concept of Performance Enhancing Proxies (PEPs) has beenintroduced. PEPs are introduced on both ends of a particular wirelessconnection and implement numerous alterations to TCP to improve theperformance over a wireless segment, without requiring modifications tothe client applications or servers. However, PEPs have at least twomajor limitations: first, they can only optimize one wireless connectionat a time; and second, they break the end-to-end semantics of ingressand egress TCP connections. UDP complements TCP in terms of reliability.However, UDP does not detect congestion in the networks, nor does itretransmit lost packets. As a consequence, UDP performance is furtherdegraded compared to TCP when used with lossy wireless networks.Furthermore, PEPs fail to optimize UDP because, unlike TCP, UDP does notestablish a connection before sending application datagrams.Consequently, conventional means of addressing such wireless datacommunications problems have been insufficient.

SUMMARY OF THE INVENTION

Principles of the invention, in illustrative embodiments thereof,advantageously provide a system for forming an optimal communicationconnection to support the needs of advanced real-time mobileapplications. Unique capabilities of the system include, but are notlimited to, an ability to dynamically leverage and aggregate individualcapabilities of all available networks simultaneously so as to form anideal “communications pipe” (e.g., an enhanced data connectionaggregating one or more capabilities of respective individual accessnetworks forming the communications pipe), thus capitalizing onperformance differences (e.g., superior bandwidth, low interference,latency, congestion, etc.) of each access network. The exemplary systemis preferably fully programmable and flexible enough to support diversegoals, whether to maximize the performance of mobile applications or toprovide stealth communications capabilities. Such benefits ofembodiments of the invention may be achieved, for example, by tightlycoupling active application profiling and inspection, policy management,multi-network scheduling, network detection and monitoring, among othermethodologies.

The enhanced data connection, according to one illustrative embodimentof the invention, is preferably constructed of a dynamic aggregation ofmultiple data connections initiated on the device. A multi-homedconcurrent data transfer is preferably provided by a multi-networktransport protocol (MNTP). The system beneficially provides a uniformand resilient network interface to unmodified software applicationswithout disrupting underlying access networks.

In accordance with one embodiment of the invention, a method for formingan optimized communication connection providing enhanced performance ofat least one application utilizing the communication connection isprovided, the communication connection including a plurality ofindividual communication networks. The method includes the steps of:obtaining a set of performance requirements corresponding to theapplication utilizing the communication connection; obtaining real-timecapacity information for each of a plurality of available channelsassociated with the respective individual communication networks;applying at least one policy-based management criteria to the availablechannels for controlling, in real-time, one or more aspects of theavailable channels; dynamically aggregating the individual communicationnetworks to form the optimized communication connection, thecommunication connection leveraging one or more features andcapabilities of at least a subset of the communication networks; andcontrolling real-time traffic scheduling across at least a subset of theavailable channels so as to adapt the communication connection tochanges in network conditions and/or policy-based management criteria.

In accordance with another embodiment of the invention, a method forimplementing a transparent concurrent multiple-network communicationwhich maintains end-to-end Internet protocol (IP) semantics andcompatibility is provided. The method includes the steps of: providing aproxy architecture between a client device and a corresponding serverdevice utilizing the multiple-network communication; intercepting, bythe proxy architecture, communications between the client device and theserver device to obtain information relating to an interaction betweenthe client and server devices in a manner which is transparent to theclient and server devices; and controlling the interaction between theclient and server devices by the proxy architecture as a function of theobtained information relating to the interaction between the client andserver devices.

In accordance with yet another aspect of the invention, a method forperforming managed data offloading using multiple access networksincludes the steps of: provisioning a client device to establishsimultaneous connectivity with the client device using at least twochannels in the multiple access networks, such that data from a firstone of the channels is offloaded to a second one of the channels inreal-time without disruption of service; adding a layer of signalingbetween the client device and at least one server device incommunication with the client device; dynamically aggregating at least asubset of available access networks as a function of decision-makingcriteria controlled jointly by a transaction between the client andserver devices; and controlling offloading of at least a portion of dataas a function of one or more characteristics of the available accessnetworks.

A system according to aspects of the invention is provided forperforming managed data offloading using multiple access networks. In anillustrative embodiment thereof, a client device (e.g., smartphone,etc.) equipped with multiple radio access technologies is advantageouslyprovisioned for performing managed data offloading as a backgroundservice and attached to a corresponding server gateway deployed in aservice provider's network. By adding a layer of signaling betweenclients and their corresponding server(s), decision-making may beperformed by a joint client/server transaction, thereby allowingnetwork-controlled management of available resources as a function ofreal-time analytics reporting.

In accordance with still another aspect of the invention, an apparatusfor multiplexing a plurality of individual network connections to forman enhanced communication connection providing at least one of increasedbandwidth, security, reliability and efficiency in a hybrid peer-to-peernetwork is provided. The apparatus includes memory and at least oneprocessor connected with the memory and operative: to discover andmaintain multiple-network connectivity for at least a subset of aplurality of respective peers forming a peer-to-peer ad hoc network; toleverage a plurality of individual multiple access networks connectedwith one or more of the plurality of peers and aggregate the pluralityof multiple access networks into a logical connection; and to implementout-of-band signaling operative to support communication and control ofthe peer-to-peer ad hoc network.

In one illustrative embodiment, a system includes a self-adaptingmechanism for robust and assisted mobile peer-to-peer networks. Thesystem ideally addresses challenges in traditional peer-to-peernetworks, such as, for example, the lack of centralized control andmonitoring, without limiting the functionality and/or advantages of suchnetworks. In a second illustrative embodiment, the system comprises amultiplexing scheme for hybrid peer-to-peer networks which preferablyprovides data path aggregation in mobile peer-to-peer networks. In thisembodiment, nodes in the system can preferably communicate directly withone another over the mobile peer-to-peer and with externalinfrastructure networks.

A multi-path transport protocol according to aspects of the invention iscapable of distributing (e.g., transmitting and receiving) multiplestreams of data simultaneously over concurrent data paths (e.g.,aggregation) between two or more multi-homed endpoints having anestablished network connection. Features of the multi-path transportprotocol (e.g., MNTP) include, but are not limited to, multi-pathconcurrent transmission, active and passive link performance probing andestimation (e.g., passive and active bandwidth and latency estimation),polymorphic scheduling support, out-of-order mitigation, configurablepayload expiration support, differentiated and granular applicationstream management, Robust Header Compression (ROHC) support, andper-link wireless optimization.

A system according to aspects of the invention utilizes multiplecognitive radios to access open, unlicensed frequency bands. The radioshop on different wireless channels and connect to different networks.One advantage of this system is to freely access available spectrum toincrease performance and reliability of the wireless network. As adirect application of this invention, upper layer applications runningon the system are oblivious to network changes caused by the radiosswitching frequencies. In this embodiment, the system beneficiallyenables applications to run undisrupted data sessions on cognitive radionetworks.

A system according to other aspects of the invention utilizes a node asa rendezvous point, e.g. peers across multiple networks exchangingresource and information through a multi-homed node. When themulti-homed node is connected to multiple networks, other nodes from onenetwork to which the multi-homed node is connected can access othernetworks to which the multi-homed node is also connected. In thisembodiment, the multi-homed node can work as a proxy or bridge amongdifferent networks.

These and other features, objects and advantages of the presentinvention will become apparent from the following detailed descriptionof illustrative embodiments thereof, which is to be read in connectionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The following drawings are presented by way of example only and withoutlimitation, wherein:

FIG. 1 illustrates at least a portion of an exemplary transparent proxyarchitecture (TPA), according to an embodiment of the present invention;

FIG. 2 is a state diagram depicting exemplary protocol-specific helperssupport for a real-time streaming protocol (RTSP), according to anembodiment of the invention;

FIG. 3 illustrates an exemplary implementation of a TPA server deployedwithin a content delivery network (CDN), according to an embodiment ofthe present invention;

FIG. 4 illustrates an exemplary methodology for bidirectional reflectionand peering, according to an embodiment of the invention;

FIG. 5 illustrates an exemplary methodology for unidirectionalreflection and peering, according to an embodiment of the invention;

FIGS. 6A and 6B illustrate exemplary methodologies for implementing TCPfreeze operations, according to an embodiment of the invention;

FIG. 7 illustrates an exemplary assisted mobile peer-to-peer network,according to an embodiment of the invention;

FIG. 8 illustrates an exemplary four-node hybrid peer-to-peer network,according to an embodiment of the invention;

FIGS. 9A and 9B illustrate an exemplary connection aggregationmethodology in a hybrid peer-to-peer network, according to an embodimentof the invention;

FIGS. 10A and 10B illustrate an exemplary connection relocationmethodology in a hybrid peer-to-peer network, according to an embodimentof the invention; and

FIG. 11 is an exemplary system in which one or more aspects of theinvention may be implemented, according to an embodiment of theinvention.

It is to be appreciated that elements in the figures are illustrated forsimplicity and clarity. Common but well-understood elements that may beuseful or necessary in a commercially feasible embodiment may not beshown in order to facilitate a less hindered view of the illustratedembodiments.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Principles of the present invention will be described herein in thecontext of illustrative embodiments of a transparent proxy architecture(TPA) for multi-link data connections that aim at optimizing theperformance of any application(s) over multiple wired or wireless dataconnections. The TPA replaces the legacy TCP protocol or any optimizedTCP implementations with a transport protocol capable of takingadvantage of a plurality of possible data connections between the proxyendpoints, while being fully transparent to legacy applications. One ormultiple network connections are available between each multi-homed TPAendpoint. Among other advantages, the TPA according to embodiments ofthe invention seeks: to optimize the performance of each of thecommunication paths between the multi-homed endpoints; to take advantageof the multiple paths between endpoints to improve performance; to betransparent to applications routing traffic through TPA; and to modifyapplications between each TPA endpoint to further optimize performance.

It is to be appreciated that the invention is not limited to thespecific apparatus and methods illustratively shown and describedherein. For example, while aspects of the invention may be described inthe context of a standard TCP/IP protocol, the invention is not limitedto such a protocol. Rather, principles of the invention may be extendedto essentially any data communications protocol (standard ornon-standard), both wired and wireless.

Aspects of the present invention will be described in further detail insections I through IV below.

I. TPA for Multi-Path Data Connections Between Multi-Homed Hosts toSignificantly Enhance the Performance of Applications Over HeterogeneousWireless Connections

FIG. 1 illustrates at least a portion of an exemplary TPA system 100,according to an embodiment of the present invention. As apparent fromthe figure, TPA system 100 preferably includes a client device 102,which may be a personal computer (PC), mobile device, etc., in operativecommunication with an application server 104, or alternative frameworkfor efficiently executing procedures (e.g., programs, routines, scripts,etc.) for supporting the construction of applications. Applicationserver 104 may be hardware-based, such as a dedicated processor, or itmay be software-based, such as program code running on a general purposeor dedicated processor and operative to perform prescribed functions.

Client device 102 is connected to a first gateway 106, or alternativerouting device, and application server 104 is connected to a secondgateway 108, or alternative routing device. The first and secondgateways are preferably operative to implement a TPA and are incommunication with one another via one or more data transport channels110. A plurality of transport channels 110 (e.g., T1, T2 and T3) may bereferred to herein collectively as a stream, and a plurality oftransport streams may be referred to herein collectively as anaggregation. The bandwidth of an aggregation is a function of therespective bandwidths of the individual channels or streams in theaggregation. First and second gateways 106, 108 are preferably operativeto control data traffic over the transport channels 110. Connections, orportions thereof, between the client device 102 and first gateway 106,between the application server 104 and second gateway 108, and betweenthe first and second gateways, may be wired or wireless, as will beknown by those skilled in the art given the teachings herein.

In the TPA system 100, client- or server-initiated connections arepreferably intercepted on a host or at a routing gateway in atransparent fashion; the content of the connections is carried over to areceiving end (the application server/client or a routing gateway on therouting path to the application server/client) using, for example, aconcurrent multi-link transport protocol (replacing TCP and userdatagram protocol (UDP), in whole or in part), or an alternativecommunication means. A good fit for this replacement transport protocolis MNTP, which is operative to perform per-connection optimizations (oralternative functions) based, at least in part, on the nature of each ofthe wireless connections, in real time.

TPA does not add any significant latency compared to a direct connectionbetween the client and the server; nor does it require modifications tothe applications. More importantly, TPA maintains the end-to-endsemantics of TCP, which a number of sophisticated protocols, such as,for example, Internet Protocol Security (IPSec), rely on. Thetransparent nature of the TPA refers to the applications initiating theconnections being essentially unaware that the connections are beingintercepted (also known spoofing or hijacking), thus ensuring fullcompatibility with legacy applications. Other proxy architectures, suchas, for example, SOCKS (which operates at Layer 5 of the Open SystemsInterconnection (OSI) model), implement explicit interception andrequire communication between the host application and the proxyintercepting the connections, explicitly requesting the proxy to performcertain actions on behalf of the host.

The interception can be performed on the host directly or on a gatewayor router on the routing path to the destination. Each approach presentsa number of benefits and drawbacks. More particularly, implementing thetransparent interception on the host eliminates the need for anadditional device to be deployed in the network (which may not be ownedor operated by the same entity owning or operating the sending host),but the interception is limited to the connections initiated or receivedby the host. Alternatively, implementing the interception on the routinggateway allows multiple hosts to benefit from the system, but can createadditional complications in the implementation of the interception.

Embodiments of the invention present several techniques for interceptingTCP, UDP and/or Internet Control Message Protocol (ICMP) connections.Some of these illustrative techniques will be presented in furtherdetail below. It is to be appreciated that the techniques describedherein are merely exemplary and are not intended to be limiting in anyway regarding the scope of the present invention, as will becomeapparent to those skilled in the art.

Socket Overloading

When an application opens a regular TCP or UDP socket, the socket callon the operating system is transparently mapped to the socket for thereplacement multi-link transport protocol implemented by the proxybetween endpoints.

TCP/UDP Splitter

When an application makes one or more socket calls, these calls areeither intercepted within the network stack of an operating system andeither redirected to a local process that accepts the connection andforwards the traffic to the endpoint over the multi-link networkprotocol, or redirected within the network stack itself.

Transparent SOCKS Redirector

When an application makes one or more socket calls, these calls areintercepted within the network stack of an operating system and mappedto corresponding SOCKS requests that are then funneled to a SOCKS proxyserver. The SOCKS proxy server, in turn, processes the requests andperforms the interception and routing of the intercepted traffic to theendpoint over the multi-link network protocol.

Virtual Interface

A “virtual interface,” as the term is used herein, is intended to referbroadly to a network interface that is supported entirely in software,which is different from ordinary network interfaces that are backed upby hardware network adapters. A virtual interface can create a newinterface to the operating system or masquerade one or multiple virtualinterfaces backed up by hardware network adapters (e.g., regular networkinterfaces). Connections sent by the operating system via a virtualinterface are preferably delivered to a user space program that attachesitself to the virtual interface, which then performs the interceptionand routing of the traffic to the endpoint over the multi-link networkprotocol.

Each technique for intercepting TCP connections presents differentadvantages and may be combined with one another or with other suitabletechniques depending upon the application, as may become apparent tothose skilled in the art given the teachings herein. For example, forimplementation of the system on a local host, the socket overloadingtechnique may be preferred, as it does not require a separate localprogram to perform the interception and is more efficient compared toother approaches. Furthermore, socket calls are usually implemented atthe kernel level in most operating systems, thereby ensuring efficientuse of resources. For implementation of the system on a gateway host,the splitter technique may be preferred, as it can be achieved without asignificant latency penalty. For intercepting traffic that reliesdirectly on the IP protocol (e.g., that is not using a session ortransport protocol—ICMP is a good example of such a protocol), thevirtual interface may be a preferred technique. Virtual interface isalso a preferred technique for applications such as virtual privatenetworks (VPNs) that may require binding to a specific networkinterface. A SOCKS redirector technique may be preferred forapplications with explicit SOCK support and specific securityrequirements.

Additional techniques for intercepting TCP connections are described infurther detail below, without loss of generality and without limitation.

Transparent Proxy Protocol (TPP)

Another major departure from traditional PEPs is the implementation of acontrol channel for communicating feedback information and/or commands,among other information, between endpoints. TPP is preferably theprotocol employed between endpoints. TPP implements atomic commands,feedback and monitoring actions that get executed on an endpoint.Non-limiting examples of such actions include the following:

-   -   Open or close a remote socket    -   Open a TCP or UDP socket and optionally bind to it    -   Connect a socket to an address of a hostname    -   Listen to a socket and expect a connection from a remote peer    -   Request a TCP or UDP port forward    -   Get/Set quality of service (QoS) requirements    -   Get/Set status requests    -   Get/Set profile information    -   Send/Request security information, usage information, etc. . . .    -   Get/Set performance metrics for a specific network interface    -   Buffer/cache/release requests specific type of traffic with a        certain size for a specific endpoint    -   Socket migration negotiation commands    -   Reflection mode synchronization

To minimize the overhead of implementing a complete control channel ontop of a transport protocol, a preferred implementation assumes thatthis information is incorporated into a header section, or alternativeportion, of the protocol used to carry payload data (e.g., piggybackingonto the data traffic being exchanged by applications). For instance, ifthe MNTP protocol is used between TPA endpoints, a reserved section ofthe MNTP header should be used.

TPA preferably implements a number of processing modules at eachendpoint to optimize operations. Those processing modules are preferablyenabled and supported by the TPP protocol.

End-to-End TCP Semantics Preservation

This technique for intercepting TCP connections preserves the TCP'send-to-end reliability and correctness semantics. The loss of end-to-endsemantics may cause problems to applications that rely on such aguarantee provided by TCP.

Header Stripping

Instead of encapsulating ingress or egress TCP/UDP frames coming from anapplication being intercepted by the TPA, TCP and UDP headers arestripped, thereby saving between about 20 and 60 bytes per IP packet forTCP, by way of example. In order to preserve end-to-end semantics, theproxy preferably uses TPP to inform the receiving end so that theheaders can be regenerated at the other (receiving) end. Headerstripping, in and of itself, can significantly improve performance whileretaining the end-to-end service model assumed by the applications.

Deep Packet Inspection (DPI) and Stateful Inspection

For QoS assignment, stateful connection interception and other usefuloptimization information passed to the multi-link transport protocol,TPA preferably includes at least two inspection modules. Statefulinspection (e.g., stateful packet inspection (SPI)) essentially onlyperforms protocol header inspection and is generally faster and lessresource-intensive than DPI. However, stateful inspection is generallymore limited in scope. At a minimum, TPA preferably performs statefulinspection, stripping the header on one TPA endpoint and regenerating iton the other endpoint. In TPA, DPI may be performed on the packetpayload to determine what the application is (and thus derives itsrequirements in terms of QoS, throughput, latency, etc.), to interceptperformance feedback used ultimately by the applications to adjust theirbehavior and used by the TPA to further optimize the connection betweenTPA endpoints, and to assist another TPA module (e.g., a protocolhelpers module) in setting up the negotiation of connections and theircorresponding interceptions.

TPA preferably implements DPI at line speed (e.g., it does not holdpackets until the DPI function is successfully executed). Rather, TPApreferably copies and holds packets in circular buffers, or alternativenon-destructive storage, that are then fed to the DPI module for furtherprocessing. This approach is a tradeoff between memory usage andconnection latency.

Protocol Helpers

For stateless protocols, such as, for example, HTTP, the TPAadvantageously does not disrupt the flow of connections and theaddressing is replaced accordingly by both endpoints. However, some moresophisticated and stateful protocols (e.g., H.323, a standardpromulgated by the International Telecommunications Union (ITU)Telecommunication Standardization Sector (ITU-T), or real-time streamingprotocol (RTSP), a network control protocol designed for use inentertainment and communications systems to control streaming mediaservers) require exchanging a series of signals and setting up multipleTCP and UDP connections between endpoints.

Since TCP and UDP connections are intercepted by TPA, it is essentiallynot possible for an endpoint to know where to direct such connectionswithout the support of the other endpoint. The situation can be furthercomplicated by the presence of NAT (Network Address Translation) and/orPAT (Port Address Translation) on the routing path. For a clientapplication and an application server to properly negotiate sessions andTCP/UDP connections through a proxy architecture, they generally requirethe support of the intercepting proxy to assist. The set of actions tobe performed is generally protocol specific, hence the need forprotocol-specific helpers.

Protocol helpers are TPA modules that are dynamically activated when aconnection implementing a stateful protocol is intercepted. For example,protocol helpers may use the TPP protocol to communicate expectedactions and commands initiated by one endpoint to another endpoint.Another illustrative use of protocol helpers is to intercept performanceinformation sent over a feedback channel between a client applicationand a server application. For instance, the RTSP reports performancebetween the client and the server in order to adjust the transmissionrate and encoding of video and audio data. That performance informationcollected by the RTSP protocol helper may, in turn, be used to furtheroptimize the communication of the multi-link transport protocol betweenTPA endpoints.

By way of example only and without loss of generality, FIG. 2 is a statediagram 200 depicting at least a portion of exemplary protocol-specifichelpers support for an illustrative RTSP protocol, according to anembodiment of the invention. While similar in some ways to HTTP, RTSPgenerally defines control sequences useful in controlling multimediaplayback, among other applications. Unlike HTTP, which is stateless,RTSP has state; an identifier is used when needed to track concurrentsessions. Like HTTP, RTSP uses TCP to maintain an end-to-end connectionand, while most RTSP control messages are sent by the client to theserver, some commands travel from server to client.

As apparent from FIG. 2, illustrative communications between an RTSPclient 202, a multi-network communicator (MNC) client 204, an MNC server206 and an RTSP server 208 are depicted. Modifications to the standardRTSP protocol according to an embodiment of the invention essentiallyenable a local MNC client 204 and MNC server 206 to function togetherfor brokering at least one connection between the RTSP client 202 andRTSP server 208. To accomplish this, aspects of the invention areoperative to essentially intercept signaling traffic between the RTSPclient and the RTSP server for performing port replacement, among otheroperations.

With reference to FIG. 2, as an illustration only and withoutlimitation, state diagram 200 shows a handshaking operation between RTSPclient 202 and RTSP server 208 which includes an RTSP sync request 210sent by the RTSP client to the RTSP server, followed by an RTSPsync/acknowledgment message 212 sent by the RTSP server to the RTSPclient and a corresponding RTSP acknowledgment 214 sent by the RTSPclient to the RTSP server in a standard manner.

Once connection between the RTSP client and server has been established,the RTSP client sends an RTSP Describe request 216 to the RTSP server. ADescribe request may include an RTSP universal resource locator (URL)(e.g., rtsp:// . . . ), and the type of reply data that can be handled.This reply may include a presentation description, typically in SessionDescription Protocol (SDP) format, although the invention is not limitedto any specific data format or protocol. Among other things, thepresentation description lists the media streams controlled with theaggregate URL.

Rather than being received by the RTSP server, however, the RTSPDescribe request 216 is first intercepted by the MNC client. Informationassociated with the RTSP Describe request is then sent by the MNC clientto the MNC server in a separate message 218. The RTSP Describe requestis then forwarded by the MNC server in a separate message 220 to theoriginally intended destination; namely, the RTSP server. Byintercepting the signaling traffic stream, the local MNC client canobtain information regarding one or more characteristics of the data(e.g., data size, format, etc.). The RTSP server then sends an RTSP OKmessage 222 to the RTSP client in response to the RTSP Describe requestconfirming receipt of the information contained in the RTSP Describerequest. This confirmation message is intercepted by the MNC clientbefore being forwarded, in a separate message 224, to the RTSP client.

Next, the RTSP client sends a Setup request 226 to the RTSP server. ASetup request, in a standard RTSP, specifies how a single media streammust be transported. This must be done before a PLAY request is sent.The Setup request preferably comprises the media stream URL and atransport specifier. This specifier typically includes a local port forreceiving RTP data (e.g., audio or video), and another port forreceiving RTCP data (e.g., meta information). In this scenario, the RTSPclient is specifying that ports A and B be used for data transmission.The RTSP Setup request is intercepted by the MNC client and theinformation contained therein is sent to the MNC server in a separatemessage 228. The MNC server performs a port replacement, substitutingports A and B with ports C and D, respectively. The MNC server thensends a modified Setup request 230 to the RTSP server specifying thatports C and D be used for data transmission. This modified Setup requestis sent to the RTSP server transparently, such that the RTSP serverthinks the request was originated by the RTSP client. The RTSP serversends a confirmation reply 232 to the RTSP client acknowledging thatports C and D are to be used for data transmission. This message isintercepted by the MNC client which sends a modified message 234 to theRTSP client confirming that ports A and B and being used for datatransport as intended. In this manner, the MNC client and MNC serverfunction together to control traffic streaming such that data sent bythe RTSP server on ports C and D is received by the RTSP client on portsA and B, respectively, without any awareness of the port replacement.This principle can be extended, in accordance with other embodiments, tovarious other communication protocols (either standard or proprietary),as will become apparent to those skilled in the art given the teachingsherein.

Signaling Bundling

Signaling bundling, as the term is used herein is intended to broadlyrefer to a concatenation of multiple data frames from one or moreintercepted TCP or UDP connections, resulting in a bigger data frame.This connection interception approach may introduce latency to theapplications sending and receiving traffic, but offers beneficialoptimizations in certain scenarios.

Consider, by way of example only, a scenario in which, on its cellularphone connected to a cellular network, an IM (Instant Messaging) usersends a message and waits a couple of seconds between messages. Once thefirst message is sent, the phone establishes a signaling path to a basestation. To preserve battery life, the cell phone may then move into anidle mode, or alternative power-conservation mode. When the user pushesanother message seconds later, the device establishes a new signalingpath. On a modern phone, many applications may be running concurrently,making constant queries to the network to push email, access socialnetworking tools and conduct other receptive actions. This constantsignaling traffic imposes a severe burden for mobile device networks(which need to allocate a channel for each such short message and thenimmediately release it—networks which were conceived for long-lived dataconnections, such as a phone call) and a battery-draining issue for cellphone users. By intelligently aggregating signaling traffic into largerframes, signaling bundling greatly reduces the battery impact andoptimizes the performance of a provider's network while only slightlyimpacting latency, an issue that may not even be perceptible to theusers depending on the class of application(s) utilized.

Selective Compression, Transcoding and Resampling

To further optimize the performance of the TCP and UDP connectionsintercepted by the TPA, selective compression and transcoding can beapplied to the traffic. Selective compression, as the term is usedherein, is intended to broadly refer to the selective application ofdata compression (e.g., by an intelligent processor or the like) basedon one or more characteristics of the data, including, for example,content type, content size and signature. Transcoding and resampling maybe defined as a server-side TPA functionality that is operative tomodify (e.g., reduce) the quality, resolution, and/or frame rate of animage or video, or alternative multimedia content, for modifying (e.g.,reducing) the size thereof. For instance, when a web page is retrieved,the HTML code can be compressed on-the-fly, a JPG image may be furthercompressed, a video may be transcoded while being streamed, and a binaryfile may be selectively skipped.

Delta Transmission

Modern data communication is very inefficient and the same binaryobjects (e.g., spreadsheets, images, audio files, etc.) are oftenretransmitted multiple times. Some applications allow for caching; forinstance, a web browser can cache certain non-dynamic elements of a webpage. However, a vast majority of modern data communication wastesprecious network resources by retransmitting the same objectsconstantly. For example, co-workers collaborating on a document mayemail one other with incremental updates to the document multiple timesper hour. With each email, the entire file gets retransmitted.

Delta transmission, as the term is used herein, is intended to referbroadly to a methodology (e.g., which may be performed in a TPA moduleor alternative processor) that detects the first instance that an objecthas been transmitted and thereafter only transmits incremental changesassociated with the object to a given destination(s). To accomplishthis, each endpoint preferably maintains an object store with an index.When a connection is intercepted, each object over a configurable sizetransmitted as part of that connection is assigned a uniqueidentification (ID), such as, for instance, a hash function, and issubsequently placed in the object store. With each subsequentconnection, heuristics are preferably used to determine the likelysimilar objects in the store. Heuristics can be based on one or moreprescribed characteristics, such as, but not limited to, file name, filesize, file type, sender/receiver identifiers, etc.

After a positive match, a delta binary diff pass (i.e., a function whichreturns a difference between files) is applied to the object and onlythe differential information is transmitted to the destination. The TPPprotocol is then used to signal the destination that a deltatransmission has been activated with a message that carries the object'sunique ID in the receiver's object store. When the delta binary diff isreceived, the corresponding object is pulled out of the object store;the binary diff is subsequently applied to the object and the originalobject is re-injected into the TCP stream, which carries it to its finaldestination.

TCP/IP Connections Migration

Migration of a TCP/IP connection preferably refers to replacing at leastone of the peers with another process that can reside on the same or ona different system. There are a number of situations in which themigration of TCP/IP connections can be useful; for instance, when thereare requirements of mobility, load balancing, QoS, fault tolerance, andsecurity. The TPA preferably includes a connection migration (ConnMig)methodology that is preferably able to migrate both ends of aconnection, does not require cooperation on both ends, is fullytransparent to an endpoint not migrating, allows an endpoint to bemigrated more than once, and may be used by a stream (as opposed tomessage) socket regardless of its internal state (even unconnectedstate—note that migrating an unconnected socks is interesting especiallyfor server-side use, e.g. migrating a socket in a listen state).

Due the nature of the TPA, it should be noted that we discuss themigration of internal multi-path transport sockets (e.g., MNTP sockets)that is the protocol used to carry the traffic of TCP/UDP connectionsthat have been intercepted by the proxy. It should be appreciated that anumber of solutions for the migration of TCP/IP connections, withoutproxy interception, already exist (e.g., M-TCP, SockMi, MSOCKS,ROCKS/RACKS, TCP Migrate, TCPcp, etc.), as will be known by thoseskilled in the art.

In the context of an illustrative embodiment of the present invention,migrating TCP/IP connections may involve one or more of the following:

-   -   migrating one endpoint or both TPA endpoints;    -   migrating one endpoint, including the Internet multi-path        transport sockets (e.g., MNTP sockets) and all TCP/UDP        intercepted connections mapped to those sockets, including, for        example, TCP/UDP connections initiated on the other end of the        proxy (for instance, with “logical” connections, such as RTSP,        that negotiates and opens multiple TCP/UDP connections on both        ends); and    -   migrating a multi-path transport socket on one TPA endpoint and        turning it into a TCP socket on another TPA endpoint.

A ConnMig process, according to an exemplary embodiment, preferablyinvolves at least two functional modules in each of the proxy endpoints;namely, a SMP (socket migration processor) module and a SMM (socketmigration manager) module. It is to be appreciated that, according toother embodiments, the connection migration methodology may compriseother or different modules for implementing the functions thereof, andthat the invention is not limited to any specific type and/or number offunctional modules employed.

The SMP module is preferably adapted for: (1) saving/restoring the stateof migrating sockets during an import/export phase; (2) exchanginginformation about migrating sockets with the SMM module; and (3)providing low-level primitives to activate and control the socketmigration facility. A primary function of the migration mechanism ispreferably to preserve the referential integrity among the datastructures that define the state of a socket. The SMP module managesTCP/UDP sockets and multi-path transport sockets in a standard fashionthrough the use of abstraction data structures, as will be known bythose skilled in the art. Besides the socket states, the SMP modulepreferably keeps track of “in-flight data;” that is, data found inreceive queues/buffers (e.g., data received by the multi-path transportsockets but not yet acknowledged (ACKed) or transmitted to correspondingTCP/UDP connections) and transmit queues (e.g., data intercepted fromthe TCP/UDP connections but not yet transmitted by the correspondingmulti-path transport sockets or not yet acknowledged).

The SMM module preferably holds sockets ready to be imported in multiplelists corresponding to one or more states of each group of sockets(TCP/UDP or multi-path transport); namely, bound sockets, listeningsockets and connected sockets. These lists change their lengthdynamically when a socket is received from the SMM module or whenanother proxy endpoint imports a socket. The SMM module is operative incombination with the SMP module to support the socket migrationmechanism.

The SMM module may carry out different tasks depending on a givensituation. For example, during an export phase, the SMM module ispreferably operative to read the state of exporting sockets from SMPmodule internal buffers. During a negotiation phase, the SMM module ispreferably operative to communicate with one or more other SMM modulesrunning on other TPA hosts in order to select where to migrate thesocket. During an import phase, the SMM module is preferably operativeto write a state of importing sockets to the SMP module internalbuffers. Complete socket migration entails searching for a TPA hostwilling to import the socket. The SMM module utilizes the TPP protocol,or a suitable alternative protocol, to communicate with other SMMmodules running on other TPA hosts.

Negotiation over TPP preferably follows a plain request-response confirmmethodology as follows:

-   -   When a proxy host A exports a socket, it sends a request in        multicast to other SMM modules running on other proxy hosts.    -   When a request arrives, a host replies provided that either the        socket is explicitly exported to that host or no specific target        proxy host is defined in the request.    -   If proxy host A does not receive a valid response within a        predefined timeout period, the migration fails.    -   The first valid response triggers a confirmation mechanism by        which proxy host A notifies all SMM modules that the socket has        been successfully migrated. Other responses arriving after the        first one are simply discarded. Other more sophisticated        policies to select a valid response can be employed, in        accordance with other embodiments of the invention; based on        rules or heuristics for instance. For example, it could be        useful to maintain statistics on previous migrations and select        a target proxy host based on achieving load balancing between        proxy hosts.

Following a successful migration acknowledgment, IP redirection ispreferably automatically handled by the multi-path transport protocol,which initiates a new aggregation with the TPA host that accepted thesocket import. Due to the nature of the TPA, application synchronizationis handled automatically.

Distributed Reflectors

One of the disadvantages of a standard TPA network topography (e.g., onethat would have a TPA client in a client device and a TPA server in aservice provider's network) is that, depending on where the endpoint isdeployed, it can cross administrative boundaries or make the networkengineering complicated. By way of example only, consider a scenariowhereby the TPA is deployed on a multi-homed mobile device equipped withtwo cellular network interfaces provisioned on two separate serviceproviders. If the other TPA endpoint is deployed outside of eachrespective service provider's network and reachable from both networks,network engineering is minimal; traffic gets transmitted through therespective networks to the TPA destination. If, however, the TPAendpoint is deployed within one provider's network, this networkconfiguration assumes that the traffic routed on the other provider'snetwork can be routed into the first provider's network, crossingadministrative boundaries. This scenario creates two potential issues:(1) cross-administrative routing is generally not an option unless bothproviders have a peering agreement in place; and (2) the solution forcesa provider to accept another provider's traffic on its network—networksthat are already saturated.

To address this issue, the TPA according to an aspect of the inventionpreferably implements distributed reflectors. In deflector mode, a TPAendpoint is only responsible for accepting a partial multi-pathtransport protocol stream (e.g., an MNTP stream) from one TPA endpointand “reflecting” it to another TPA endpoint. The reflecting operationessentially means forwarding data traffic from one stream and syncing upits state with the destination TPA endpoint.

Different policies can be applied to reflections, which can beunidirectional or bidirectional. Policies can be set based on prescribedcharacteristics, such as, for example, type of traffic, value-addedoffering, load balancing and traffic engineering policies.

FIG. 3 illustrates an exemplary system 300 for implementing distributedreflectors, according to an embodiment of the invention. The system 300includes a client device 302 and an application server 304 operativelycommunicating with one another via a content delivery network (CDN) 306.As shown in the figure, a natural place to deploy a TPA server 308 iswithin the CDN 306 or within the provider's network with the TPAendpoint deployed within the CDN or on the application server itself.System 300 employs two reflectors 310 and 312, configured in adistributed manner, although the invention is not limited to anyspecific number of reflectors.

FIG. 4 illustrates an exemplary methodology for performing bidirectionalreflection and peering, according to an embodiment of the invention. Inthis embodiment, a system 400 employs two reflectors 410 and 412 havinga bidirectional communication channel 411 established therebetween. Oneof the reflectors 410 may be controlled by a first TPA 414 and the otherreflector 412 may be controlled by a second TPA 416, as shown.

FIG. 5 illustrates an exemplary methodology for performingunidirectional reflection and peering, according to an embodiment of theinvention. In this embodiment, a system 500 includes a reflector 502associated with a first operator (Operator 1), the reflector having aunidirectional communication channel 505 established with a first TPA504 associated with a second operator (Operator 2). The reflector 502and TPA 504 each are in bidirectional communication with a second TPA506 associated with client device 302.

Smart Caching and Buffering

The TPA according to embodiments of the invention is preferablyoperative to work closely with a multi-path transport protocol (e.g.,MNTP), connecting its endpoints to optimize the transmission ofapplication data traffic. The multi-path transport protocol aggregatesthe capabilities of multiple access networks simultaneously to the sumof each individual bandwidth available. If the access networks arewireless, the total available bandwidth can vary widely due theshort-lived nature of some wireless networks (for instance, userstraversing Wi-Fi networks), signal fading or link errors, and the natureof the technology itself (for instance, a cellular connection maydegrade from its 3G variant, to its 2.5G or 2G variant, cutting theobservable throughput by as much as 300%). As a consequence, thebandwidth available to applications taking advantage of the TPA cancontinuously change. For some applications, such as, for example,streaming video, a drastic change in available throughput may lead tointermittent interruption of service as the buffer on the receiving endwaits to be filled. Newer streaming technologies have introducedadaptive bit rate and compression based on available bandwidth, which isdetermined by periodic probes sent by the streaming server or based onfeedback periodically exchanged on a parallel control channel, or analternative means for determining available bandwidth at any given time.Similarly, certain streaming technologies offer multiple streams encodedfor different classes of link performance and are able to switch streamsdynamically to provide the best experience based on instant throughputavailable between peers.

While these techniques allow a client to recover faster frominterruptions of service, it is not an optimal solution. In accordancewith aspects of the invention, the TPA further improves performance byintroducing the concept of smart buffering. Using techniques of smartbuffering, the TPA utilizes its direct knowledge of instant bandwidthavailability and capability, concurrent stream count and impendingchanges to the set of aggregated access networks (e.g., reported by themulti-path transport through a socket API or alternative means) tobeneficially adjust the amount of caching on behalf of the applications,in an intelligent manner, tracking the variance in link performance.

The TPA, according to embodiments of the invention, preferably alreadyimplements buffers to store data intercepted from TCP/UDP connectionsand fed to the multi-path transport protocol on one end and retrievedfrom the multi-path transport protocol and fed to the TCP/UDPconnections on the other end. For smart caching, both proxy endpointsare adapted to implement additional dynamic buffers mapped to specificapplication streams (e.g., intercepted TCP/UDP connections).

Consider the following illustrative scenario wherein client applicationA running a transparent proxy B locally communicates to a streamingserver D through a proxy endpoint C (e.g., running either on a streamingserver or on a routing path to an application server). When client Arequests a video from server D, the connection is accepted locally byproxy B and its payload is routed to proxy endpoint C over themulti-path transport protocol and a new connection is established toserver D. When server D starts streaming, the video traffic flows toclient A, through the proxy [C, B]. When proxy C is alerted that theperformance of the [B, C] link is about to change, it transmits anotification to proxy C (e.g., over TPP) containing the expectedbandwidth improvement, degradation, or offline time. Upon reception ofthe notification, proxy C immediately requests additional video framesfrom server D to fill up its local caching buffer and immediately startstransmitting to proxy C for storage in the remote caching buffers, readyto feed the intercepted TCP/UDP connections of client A, therebyensuring the best playback and concealing the variance in linkperformance in a transparent manner. Similarly, each proxy endpoint mayrequest that a specific object (e.g., video, file, etc.) be cachedremotely on its behalf for an amount of time (prescribed or arbitrary)in order to optimize certain operations, such as, for example, a filedownload. Once the operation (e.g., file download) has completed,transmission of the specific object can resume as normal.

Similarly, when more than one client (say, for example, clients A and E)are accessing the same remote content around the same time on server D,proxy C can pre-fetch the content from server D and cache it locally.Proxy C then supplies the content to clients A and E from its localcache. This method saves bandwidth between server D and proxy C when thecached content is requested many times (e.g., a popular video file).This method is also applicable when the content is dynamicallygenerated.

Smart caching and buffering, when used in concert with stack freezing(described in further detail below), allows essentially instantreconnection and resume operations, regardless of the duration of theinterruption (e.g., a few seconds or a couple months). Furthermore,smart caching can be used to preload content in an intelligent mannerfor web browsing (e.g., fetching links proactively) or audio streaming(e.g., fetching a next track proactively).

TCP Stack Freezing

The TPA, in accordance with embodiments of the invention, preferablyworks closely with the multi-path transport protocol (e.g., MNTP),connecting its endpoints to optimize the transmission of applicationdata traffic. The multi-path transport protocol aggregates thecapabilities of multiple access networks simultaneously to the sum ofeach individual bandwidth available. The TPA, in an embodiment of theinvention, includes a TCP stack freezing (TSF) module, or alternativeprocessor, operative to sustain application sessions even throughsuspend-and-resume cycles and loss of connectivity.

FIGS. 6A and 6B illustrate exemplary state transition diagrams 600 and650, respectively, depicting implementation of TCP freeze operations,according to an embodiment of the invention. The system preferably usesa feature implemented in standard TCP implementation in that when areceiving end host does not have adequate buffer space left, it informsthe sender to stop sending more packets.

By way of example only and without loss of generality, the basic conceptis to freeze all TCP timers when the receiving window is full. When asender transmits a packet, the receiver sends back an acknowledgment(Ack) of receipt of the transmitted packet along with an updated windowsize, for instance, enough to store four more packets. The sender thensends four more packets as allowed by the specified window size (Win=4).When the receiver has received all four packets but does not want thesender to transmit any more packets, it sends an acknowledgment of thelast received packet with an updated window size of zero (Win=0). Suchan acknowledgment is referred to herein as a zero window advertisement(ZWA), as shown in FIG. 6B.

After receiving the ZWA, the sender knows it cannot send any morepackets but instead of terminating the connection, the sender enters apersistence mode and periodically sends a probe packet to the receiver.The probe packet may comprise, for example, a “heartbeat” included inthe TCP protocol. This packet is referred to herein as a zero windowprobe (ZWP), as shown in FIG. 6A. After receiving the ZWP, the receiversends the ZWA to the sender with the same acknowledgment number and awindow size of zero. This process repeats thereby maintaining theconnection. When the receiver wants to continue the transmission, itsends the acknowledgment with an updated non-zero window size, andnormal operation is resumed.

The TPA system is already capable of intercepting TCP connections onboth endpoints. To perform stack freezing, an additional TSF module isimplemented on either or both sides. When connectivity is interruptedbetween both ends of a TPA system, the TSF module sends a ZWA or ZWP tothe local host TCP as required to keep the TCP connection active. Inaddition to sending a ZWA or ZWP, as appropriate, the TSF module ispreferably also responsible for tracking and storing the correctacknowledgment number for each TCP connection.

II. Multi-Path Transport Protocol Capable of Distributing Traffic OverMultiple IP Networks Simultaneously

MNTP (multi-network transport protocol) is a connection-orientedprotocol operating at a transport layer of the OSI model. It is designedto transmit and receive multiple streams of data simultaneously overconcurrent data paths (e.g., an aggregation) between two or moremulti-homed endpoints that have established a network connectiontherebetween. MNTP is especially designed to manage and optimize networkconnections established over wireless links.

As an initial matter, MNTP will be described herein using the followingnon-limiting definitions for certain terminology used in connectiontherewith.

Endpoint

In the context of MNTP, an endpoint may be broadly defined herein as alogical sender/receiver of MNTP packets. An endpoint characterizes thesending or receiving side of an aggregation. On a multi-homed host, anMNTP endpoint may be represented to its peers as a set of eligible boundaddresses which the MNTP packets can be sent to and/or received from.The endpoint also preferably holds a list of aggregations that itrepresents.

Aggregation

An aggregation may be broadly defined herein as a connection between twoor more endpoints. It is visible from an application level of the OSImodel. Aggregation can be either a point-to-point type, where exactlyone aggregation peers to exactly one other aggregation, or a one-to-manytype, where one server aggregation peers to N endpoints, where N is aninteger greater than one. In the latter case, the aggregation supportsthe concept of broadcast messages (e.g., one message sent to multipleendpoints) and the peeling-off of an aggregation where a point-to-pointdedicated aggregation is created between two peers that were in aone-to-many aggregation.

Stream

In the context of MNTP, a stream may be broadly defined herein as asequence of user messages that are to be delivered to an upper-layerprotocol (e.g., though an MNTP socket API) in order relative to othermessages within the same stream.

Transport

A transport represents a link between a pair of endpoints' boundaddresses IP:PORTs tuples. Each transport has a local address:port and apeer address:port. Each transport independently manages its owncongestion control.

Packet

Messages between MNTP endpoints may be referred to as MNTP packets. Eachpacket preferably comprises an MNTP header followed by one or many MNTPchunks.

Chunk

A chunk is contained within an MNTP Packet. A chunk as used herein isintended to be defined broadly as a fundamental unit in informationexchange among MNTP aggregations. Based on their functions, chunks canpreferably be categorized into two types; namely, CONTROL chunks andDATA chunks. A CONTROL chunk carries theadministration/management/configuration information and might triggerMNTP change from one state to another. For DATA chunks, the payloadwithin a DATA chunk represents a piece of one message from onecorresponding application stream.

Socket

An MNTP socket may be broadly defined herein as an interface that anapplication uses to communicate with the MNTP protocol. The features andfunctions of MNTP are represented to the application in the form ofsocket application program interfaces (APIs). An MNTP socket keeps areference to the corresponding endpoint it is bound to and a queue tobuffer the messages transferred through MNTP before they are passed tothe applications. The MNTP socket is also characterized by a set ofsocket parameters.

MNTP is a connection-oriented protocol that aims to be flexible andpreferably includes one or more of the following features:

Multi-Path Concurrent Transmission

MNTP implements concurrent transmission of data traffic from a source toa destination host via two or more end-to-end data paths. A host ismulti-homed if it can be addressed by multiple IP addresses. When bulkdata from an application (or applications) is sent to MNTP, it is slicedup into multiple chunks and passed to a scheduler engine, or alternativeprocessor, that selects a transport to send the chunk. Chunks queued fora specific transport are concatenated in an MNTP packet and transmittedto the destination over the network interface corresponding to thetransport.

Reasons for implementing multi-path concurrent transmission are numerousand include, but are not limited to, improving the bandwidth availableto applications, reducing latency for real-time, latency-sensitiveapplications, increasing link reliability for an application (e.g.,using data mirroring), or even special facilitating communication modefor applications (e.g., LPI—low probability of intercept) or increasingsecurity for the applications.

Active and Passive Link Performance Probing and Estimation

MNTP is preferably operative to estimate the performance (e.g.,bandwidth, round trip delay time (RTT), jitter, etc.) of each transportin order to optimize the scheduling and concurrent transmission of datatraffic. Failure to accurately estimate the available bandwidth andlatency can result in sub-optimal aggregated bandwidth and may introducelarge out-of-ordering which further reduces throughput seen by theapplications. MNTP introduces multiple techniques to estimate paththroughput and delay. Such estimation is performed either passively(e.g., measurements on transmitted traffic) or actively, for exampleusing special probes. Active probing is more accurate but generally addsoverhead complexity and wasted bandwidth.

Passive Bandwidth and Latency Estimation—

Since MNTP is adapted to transmit data over multiple data pathsconcurrently, it can use the data sent and acknowledged to measure RTT,smoothed RTT (SRTT) and RTT Variation (RTTVAR) on a per-path basis.Passive and non-intrusive bandwidth is achieved using low-pass filteringon a rate of returning MNTP selective acknowledgments to estimate thebandwidth available for that connection.

Active Bandwidth and Latency Estimation—

MNTP is operative to implement an active probing scheme which may bereferred to herein as probe train exponential pattern probing (PTEPP).PTEPP uses a novel exponentially-spaced train of UDP probes sent with anexponential packet flight pattern strategy to dynamically estimate theavailable bandwidth along an end-to-end network path. PTEPP is designedto be highly efficient (wasting less bandwidth); when estimating anetwork path over the range of rates [R1, R2]Mbps, PTEPP only useslog(R2)−log(R1) packets. Each UDP packet preferably carries a sendertimestamp, or alternative chronological identifier, which the receiveruses along with its own local timestamp in the delay estimation process.Probe packets preferably travel one-way, from sender to receiver, andthe receiver performs the estimation.

Support for Polymorphic Scheduling

MNTP, according to embodiments of the invention, preferably implements anovel scheduling scheme, referred to herein as polymorphic scheduling. Apolymorphic scheduler is a scheduler that is adapted to reconfigureitself based, at least in part, on one or more characteristics of theconnections it aggregates and the traffic requirements imposed byapplications. By default, MNTP implements a bandwidth-aware schedulingwith the objective of maximizing a ratio of in-order delivery overmultiple paths. Bandwidth estimation on paths that have not been usedover a certain period of time can be performed using the PTEPP scheme,while bandwidth estimation on active paths is preferably performed usinga passive low-pass filtering technique, or alternative bandwidthestimation means.

Scheduling is preferably performed on the sender side only usingavailable window size, latency estimates, bandwidth estimates andin-flight and queue traffic computations. However, some applications mayrequest specific scheduling strategy. For instance, an application mayrequest traffic to be mirrored in order to increase reliability orquality of a video stream; another application may request that itstraffic be mapped onto a specific path for security reasons, etc.Additionally, the nature of the access networks involved may influencehow the ingress traffic is being scheduled. For instance, retransmissionand acknowledgments are better sent on a 3G cellular interface ratherthan using a Wi-Fi interface to avoid self-contention issues. Onelow-bandwidth but reliable link may work in conjunction with ahigh-bandwidth but lossy link to improve performance; links may havedifferent pricing considerations and applications may request a low-costrouting. This is in addition to specific QoS mapping requirements,quotas and priorities and the like assigned by applications. As such,the MNTP polymorphic scheduler is able to reconfigure itself to reflectthose requirements.

In addition, according to embodiments of the invention, MNTP supports a“hot-swap” (i.e., loading and unloading while data is being transmitted)scheduling mechanism allowing pre-determined and self-containedscheduling strategies to be deployed—upon changes in conditions orrequirements—even in the presence of in-flight traffic. A good exampleof a scheduling mechanism/strategy is a LPI (low probability ofintercept) scheduler. When communication is established over a singlewireless link, it is possible to triangulate the location of thesender/receiver. An LPI scheduler transmits sequential parts of alogical communication flow using randomly selected data paths. In thismanner, instead of being transmitted concurrently over multiple datapaths, the traffic is transmitted sequentially over a single data paththat changes randomly, at random intervals.

Out-of-Order Mitigation

As is the case with TCP, reordering introduced in an MNTP flow generallydegrades throughput. When multiple paths used for concurrent transfershave different delays and/or bandwidth characteristics associatedtherewith, a sender can introduce significant packet reordering in theflow. Reordering is an inherent consequence of concurrent multi-pathtransfer and is difficult to completely eliminate in an environmentwhere the end-to-end path characteristics are constantly changing or areunknown a priori, as in the case of a wireless network, for example.MNTP, according to embodiments of the invention, preferably introducesat least two fundamental techniques to mitigate out-of-ordering effect:displacement frequency map (DFM) and reordering density map (RDM); andreordering storm avoidance (RSA), details of which will be presentedherein below.

Displacement Frequency Map (DFM) and Reordering Density Map (RDM)—

RDM may be broadly defined herein as the distribution of displacementsof packets from their original temporal positions, normalized withrespect to the number of packets. An early packet corresponds to anegative displacement and a late packet corresponds to a positivedisplacement, although such assignment is essentially arbitrary. Athreshold on displacement may be used to keep the computation withinprescribed bounds. The choice of threshold value may be a function ofthe measurement uses and constraints, such as, for example, whetherduplicate packets are accounted for when evaluating these displacements.

DFM can be viewed as a normalized histogram of the occupancy of ahypothetical buffer that would allow recovery from out-of-order deliveryof packets. If an arriving packet is early, it is added to thehypothetical buffer until it can be released in order. The occupancy ofthis buffer, after each arrival, is used as a measure of reordering. Athreshold, if employed, may be used to declare a packet as lost, therebykeeping the complexity of computation within certain bounds. Thethreshold may be selected based on application requirements insituations where the late arrival of a packet makes it useless, e.g. ina real-time application. Both DFM and RDM are computed at the receivingend. MNTP preferably uses its signaling functionality to providefeedback on reordering to the sender, which, in turn, may use suchinformation to modify (e.g., optimize/refine) its schedulingaccordingly.

Reordering Storm Avoidance (RSA) Scheme—

Reordering can cause a number of side effects that are preferablyaddressed by RSA, including, for example:

(a) unnecessary fast retransmissions (e.g., when reordering is observed,a receiver sends gap reports to the sender which uses the report todetect loss through a fast retransmission procedure. However, thiscauses two issues: namely, each retransmission is assumed to occur dueto a congestion loss, the sender reduces its congestion window for thedestination on which the retransmitted data was outstanding and acongestion window overgrowth problem causes a sender's congestion windowto grow aggressively for the destination on which the retransmissionsare sent, due to acknowledgments (ACKs) received for originaltransmissions. MNTP is operative resolve this issue, for example byimplementing virtual queues per destination within the sender'sretransmission queue, applying the retransmission procedure on aper-destination basis without affecting the sender's behavior);

(b) unnecessary congestion window reduction (e.g., a congestion windowalgorithm in MNTP allows growth in a congestion window when a newcumulative acknowledgment is received by a sender. When selectiveacknowledgments (ACKs) with unchanged cumulative acknowledgments aregenerated due to reordering and later arrives at the sender, the senderdoes not modify its congestion window, assuming the loss is due tocongestion and not reordering. Since a receiver naturally observesreordering, many selective acknowledgments are sent containing new gapreports but not new cumulative acknowledgments. When these gaps arelater acknowledged by a new cumulative acknowledgment, congestion windowgrowth occurs but only for the data newly acknowledged in the mostrecent selective acknowledgment. Data previously acknowledged throughgap reports will not contribute to congestion window growth. MNTP solvesthis issue by implementing a congestion window algorithm that usesknowledge of transmission destination to deduce in-order delivery perdestination and uses selective acknowledgments, which acknowledge thisdata to update the corresponding congestion window); and

(c) a common design for transport protocol used in the Internet is tosend an acknowledgment packet after receiving two DATA packets to reduceacknowledgment traffic in the Internet, thereby saving processing andstorage at routers on the acknowledgment path. However, reordered DATApackets need to be acknowledged immediately. Due to frequent reordering,this can often cause a receiver to not delay acknowledgments. To preventthis increase, MNTP preferably always delays acknowledgments,irrespective of whether or not data is received in order.

Configurable Payload Expiration Support

MNTP provides partially reliable service—e.g., a service that allowsapplications to specify, on a per-message basis, rules governing howpersistent the transport should be in attempting to send a message tothe receiver. MNTP preferably defines multiple levels of partiallyreliable services: expiration, buffer size and number ofretransmissions.

When one or more messages are expired, MNTP preferably notifies its peerendpoint using a specific control chunk containing sequence numbers ofthe expired messages. The receiving side treats the messages as if theywere received. Additionally, the control chunk may contain informationabout the transport on which the message should have been sent.

In one embodiment of the invention, MNTP fully supports configurablepayload expiration on one or more streams composing the aggregationformed by MNTP. In particular, MNTP preferably employs one or moreunique congestion algorithms to handle expired messages. The congestionalgorithms are operative to ensure that sent but unreceived messages arenot counted towards congestion control based decision-making. MNTPfurther facilitates the notification of expired messages to avoidblocking of in-order delivery messages.

Differentiated and Granular Stream Management

MNTP, according to an embodiment of the invention, supports the conceptof multiple streams within an aggregation. Streams can be configured tohave different properties, including but not limited to reliability,reconfigurable payload expiration, priority, bandwidth requirement,transport and/or channel binding. Within a single aggregation, MNTP canpreferably apply different scheduling schemes to provide granular anddifferentiated management.

Expiration-based service allows applications to specify that sometraffic is highly sensitive to timeliness. For instance, it isessentially meaningless to transmit VoIP packets after some prescribedperiod of time has elapsed. Buffer size service allows applications tospecify how much of the data can be stored in a buffer before beingrecycled (i.e., overwritten) with new data. Retransmission-based servicecan provide a connected, ordered, but reliable data transfer service, aservice very desirable for video transfer for instance.

Support for ROHC (RObust Header Compression)

In accordance with embodiments of the invention, MNTP fully supportsROHC, which reduces the size of its header (e.g., from 32 bytes to 3bytes, although the invention is not limited to any specific number ofheader bytes or reduction of header bytes). To accomplish this, MNTPpreferably incorporates a compressor on its sending queues and adecompressor on its receiving queues, or alternative processing means.

Per-Link Wireless Optimization

In accordance with embodiments of the invention, MNTP implementslink-specific algorithms for congestion window growth, congestion windowreduction, back-off algorithm and/or slow start algorithm; this schemeallows MNTP to further optimize the transmission over each underlyingwireless access network. This optimization is preferably able to work inconjunction with the active and passible link performance probing andestimation.

III. Granular Mobile Offloading System for Mobile Devices

Smartphones are driving tremendous increases in mobile data usage,straining mobile networks in the process. As a result, mobile operatorsare finding it necessary to dramatically increase network capacity whileconcurrently meeting prescribed performance requirements of theirsubscriber base. Scaling network capacity using traditional means is anexpensive proposition, and thus undesirable. Furthermore, this approachmay not even be achievable in some cases; for instance, in very denseareas where radio frequency (RF) spectrum scarcity combined withpropagation issues at some frequencies make the solution impractical. Inaddition, it is estimated that about 70 percent of all mobile datatraffic is migrating to the Internet, and under current networkarchitectures, that traffic hops over several core network nodesunnecessarily to reach its final destination.

Two classes of solutions have emerged to alleviate the capacity issue ina cost-effective manner for operators: Wi-Fi and femtocell offload, andoffload gateway.

Wi-Fi and Femtocell Offload—

The widespread availability of Wi-Fi, installed in millions of homes andoffices around the world, as well as in many smartphones themselves,make Wi-Fi a natural technology choice for mobile operators. Wi-Fiaccess points and hotspots can ease 3G capacity constraints in a radioaccess network, as well as in backhaul and core networks, by divertingInternet traffic from the mobile core and sending it directly to theInternet (sometimes referred to as local IP breakout). This methodologycan beneficially save enormous data plane capacity for the operators'own services or for third-party services to which it can add value. Inaddition, vendors of femtocells (tiny home or office base stations) havelong touted data offloading as a key benefit for operators.

Offload Gateway—

As previously described herein, most of the offloading efforts havefocused on offloading data traffic from the radio network using, forexample, fixed mobile convergence (FMC) technologies, including, but notlimited to, Wi-Fi and femtocells. Recently, however, a group of vendorshas emerged offering solutions to unburden operators' core networks aswell, by offloading Internet-bound traffic before it ever sees its firstcore gateway. So-called “offload gateways” can sit between an RNC (radionetwork controller) and an SGSN (service gateway support node), behindthe core directly or directly in the ASN (access network node) gatewayfor a Wi-MAX network, GGSN (GPRS gateway support node) for a HSPAnetwork or on the data and signaling gateways for LTE EPC (long-termevolution—evolved packet core). Regardless of the positioning in thenetwork, the objective remains the same: redirecting Internet-bound datatraffic off a carriers' mobile core.

While it may help operators reduce the load from their core networks,both approaches presented above exhibit certain drawbacks. In the caseof Wi-Fi and femtocell offload, in order to maximize the use of Wi-Fioffloading, handset vendors force the switchover from the cellularnetwork to Wi-Fi, which, in most cases, breaks existing connections.Protocols such as, for example, IMS (Internet Protocol MultimediaSubsystem), Mobile IP and UMA (Unlicensed Mobile Access), can somewhatreduce the issue, but they either need the Wi-Fi and cellular networksto be operated by the same operator (e.g., in the case of IMS), do notwork for real-time traffic such as voice and video (e.g., in the case ofMobile IP), or are missing handset support (e.g., in the case of UMA).

Complete network bypass can be achieved by putting a small applicationon a subscriber's device which is operative to detect when the device isin a Wi-Fi area and automatically move all the data access to thatnetwork. Voice may continue to be delivered via the core network. Thisapproach, however, has some major drawbacks as well. First, the carrierloses visibility and control of their subscribers while they are in theWi-Fi area. Second, since there is no connectivity between the corenetwork and the device, the carrier is unable to deliver any 3G content,leading to potential loss of revenue. Third, voice service will soontransition from circuit-switched networks to IP networks. If voice isbypassing the core network, the operator is prevented from billing forusage, in addition to voice call termination and routing issues. Fourth,the Wi-Fi network that the subscriber is forced onto may have worseperformance than the cellular network and the operator would have novisibility with an unmanaged Wi-Fi network to ensure that the QoS can bemaintained or improved.

In the case of offload gateways, such alternative solution is notnecessarily a panacea for operators either. Offload gateways are able todetermine if a session is Internet bound and if the operator has novalue to add to the transaction, the gateway shunts the traffic streamoff the carrier network onto the Internet, thereby avoiding the multiplewaypoints it would normally travel through in the carrier's core.However, offload gateways are yet another piece of equipment to deployand an expensive proposition, as they are very resource-intensive due totheir need to perform traffic inspection and decision-making at linespeed. Additionally, offload gateways do not resolve the signaling stormissue caused by constant, short-lived and periodic updates made bymobile devices.

Accordingly, aspects of the invention provide a system that performs anadvanced, granular and managed offloading mechanism using multipleaccess networks. In accordance with an illustrative embodiment of theinvention, a client device, such as, for example, a smartphone oralternative device, equipped with multiple radio access technologies isprovisioned with a Dynamic Polymorphic Capacity Concatenation (DPCC™, atrademark of Attila Technologies LLC) system running on the clientdevice as a background service and attached to a corresponding servergateway deployed in a service provider's network, ideally on an ASNgateway directly. The DPCC system provides connectivity while roamingbetween access networks and across administrative domains, adapting thenature of the aggregated connection to reflect changes in the networkenvironment, application needs, security levels, or user preferences.Thus, DPCC provides multi-network capability, which can be applied tomobile routers, laptops, and handheld devices, and further enhances thewireless security and bandwidth for video, interactive collaboration,gaming, and surveillance, among other applications. The client device isavailable to leverage a plurality of interfaces to maintain optimum QoSfor applications running on the device while being able to performgranular and network-controlled offloading.

In a regular mode of operation, a client utilizes and leveragesaccessible networks simultaneously to essentially build a dynamicnetwork connection that matches the capabilities of the differentnetworks to the needs of applications in order to maximize theirperformance and the overall experience for the subscribers. This dynamicnetwork connection has characteristics that are superior to any one ofthe individual accessible networks. The default real-time,decision-making operations on the client are constrained by policiesthat are distributed locally or by a server entity.

By adding a layer of signaling between clients and the server, thedecision-making becomes a joint client/server transaction, allowingnetwork-controlled management of available network resources based onreal-time analytics reporting or other criteria. Advantageously, theprocess can be entirely client-driven or network-controlled. In aclient-driven context, the client may contain local policies thatdescribe default expected behavior for specific access networks, RFperformance or application streams, etc. A decision engine dynamicallyaggregates a set of available access networks so as to optimize theperformance of the devices (e.g., QoS, performance of applications,battery usage, etc.). In a network-controlled context, a serverpreferably instructs the client on how available access networks shouldbe utilized based, at least in part, on network performance, servicelevel agreement (SLA), premium services quality assurance, applicationrequirements, and/or other characteristics.

The client relies on the TPP protocol to periodically report its uniqueID, reachable access networks (based on radio events, backgroundscanning or location information), location, network performance report,application streams and expected QoS to the server. The servercommunicates with the clients to enforce specific multi-path routingpolicies, send authentication credentials to establish data sessions onroaming partners' networks, update lists of reachable networks in ageographical area (limiting periodic background scanning on the device,a battery draining process), etc.

A system according to embodiments of the invention preferably supportsone or more of the following functionalities:

Client-Activated and/or Network-Controlled—

The system leaves the client with the flexibility to perform theoperation that will make the best use of its resources but can beactivated from the supporting network in order to optimize itsoperation. Network control of the offloading is based, at least in part,on a transaction between the client reporting its network capabilitiesand the application and server responding with a policy-based decision.

Foreign Networks—

The client can offload traffic onto networks not controlled by theservice operator the client is subscribed to (i.e., foreign networks)without requiring special agreements between both network operators.

Granular Offloading—

Offloading can be performed on all or a portion of the traffic,reflecting the capabilities of the network it is being offloaded onto(e.g., 40 percent of the total traffic can be offloaded). The offloadingcan be based, for example, on data type (e.g., only video frames areoffloaded), application type (e.g., bandwidth-hogging applications),time-based, QoS-based (e.g., premium services), location-based,load-based (e.g., network load at the RNC), power-based (e.g., batterylevel on the device), total cost of services, or other criteria assuitable for a given scenario, as well as the aggregated considerationof multiple of such criteria.

Cellular Signaling Tunneling Support—

The system supports the tunneling of cellular signaling over IP allowingmobile core avoidance but keeping the relationship subscriber/providerintact.

Signaling Bundling Support—

The system supports bundling of background data signaling over anetwork-specified time window, greatly reducing the transient channelallocation on the RNC and optimizing the battery life on mobile device.

Connectivity Transaction—

The system supports a periodic location beacon system that informs theserver of the client's location. The server responds with a list ofreachable networks at the location, thereby avoiding battery-intensivebackground scanning on the client host.

Analytics and Usage Statistics—

The system provides the ability to report user-centric analytics andmetrics for coverage and connectivity to the server host, therebyallowing mobile operators to use multi-dimensional intelligence toimprove the mobile broadband experience and to increase revenue from themobile broadband services.

By way of example only and without loss of generality, the followingillustrative scenarios demonstrate the applicability of the system inreal-world conditions, according to embodiments of the invention:

Scenario #1:

A cellular subscriber enters a Wi-Fi zone in a coffee shop where theWi-Fi network is not operated by the cellular provider. The subscriberis running a real-time entertainment application on the device streamingvideo from an Internet website not operated by the cellular provider.Before entering the Wi-Fi zone, the application was using the 3 Gservices from the operator to stream traffic. Upon detecting a new radioevent, the client communicates with the server deployed in the carrier'snetwork and reports its location, availability of one or more Wi-Finetworks and their reported performance, the applications running on thedevices and their QoS requirements. The server responds withinstructions to offload all data traffic to the Wi-Fi network. Theclient seamlessly offloads the traffic to the local Wi-Fi network,preserving the ongoing network sessions and directly to the Internetserver (e.g., by migrating the connection endpoint locally), thuscompletely bypassing the operator's core network. The 3G radio is put ina dormant mode, preserving the battery on the device. All the 3Gsignaling and cellular-based service (e.g., SMS/MMS) are still funneledthrough the carriers' network core. After some time, as the subscriberleaves the Wi-Fi zone, the client automatically reactivates the datasession on the 3G network and for a short period of time, the traffic issplit among the two active data connections, until the Wi-Fi connectiondrops and is entirely removed from the network aggregation.

Scenario #2:

Assume a scenario similar to scenario #1 above, but the application is apremium service distributed by the operator (e.g., a MobileTVapplication). Many paid applications distributed by service providersrely on the inherent built-in security mechanisms provided by thecellular service to restrict unauthorized use of the application.Current offloading solutions would not work in this instance because theentire application is inherently tied to the cellular network it needsto communicate with in order to operate. In contrast, embodiments of theinvention preferably enable the client to peer into the traffic streamand dispatch the security traffic over the 3G connection and retrievethe video frames over the Wi-Fi network, concurrently andsimultaneously, removing a large portion of the traffic off thecarrier's core network while keeping the application secure and therelationship with the subscriber intact.

Scenario #3:

Assume a scenario similar to scenario 1 above with a Wi-Fi networkoperated by a roaming partner with which the operator has established apeering agreement. The subscriber is still using the premium streamingapplication of scenario 2. Upon detecting the Wi-Fi network, the clientupdates the server via TPP signaling. The server responds with thesecurity code to connect to the Wi-Fi network and instructs the clientto offload all traffic to the 3G network (including all cellularsignaling). Since the cellular operator has a peering agreement with theWi-Fi operator, all 3G signaling is encapsulated and uses directtunneling (e.g., I-HSPA) to backhaul it to the carrier's core networkwhile all Internet-bound data is routed directly to it. This scenario isespecially appealing when subscribers are roaming outside the primarynetwork and greatly reduces roaming cost by bypassing the core networksof roaming partners.

IV. Multiplexing Network Connections for Efficient, Secure and ReliableMobile, Hybrid Peer-to-Peer Network

With the ever-increasing demand for connectivity, wireless communicationhas gained rapid acceptance. The use of mobile devices, such as, forexample, laptops and smartphones, has become prominent in daily life anda vision for nomadic and ubiquitous computing has spurred innovation. Todate, most wireless technologies are infrastructure-based; i.e., theyrely on the support of base stations or access points. However, thesetechnologies are not well-suited in some situations such as militaryoperations, search-and-rescue operations and the like, where access toinfrastructure may be limited or entirely unavailable. Yet, theseoperations require rapidly deployable communications with reliable,efficient and dynamic networking.

Mobile peer-to-peer networks (sometimes referred to as mobile ad hocnetworks or MANETS) have been helpful in bridging that gap. Peer-to-peernetworks are dynamic, infrastructure-less networks in whichparticipating nodes are required to collaborate with their peers inorder to support the formation of the network as well as its operation.Although these types of networks have gained traction in recent years,they are yet to be deployed on a larger scale, due at least in part tothe fact that none of the proposed architectures and protocols to datesufficiently satisfies the requirements for wider acceptance (e.g.,reliability, efficiency and security).

Security is a challenge for mobile peer-to-peer networks. As an open andcollaborative environment, nodes need to cooperate with their peers insupporting the network functionality. Consequently, in practice any nodecan maliciously or selfishly disrupt or deny communication to othernodes. Heavy security schemes relying on strong cryptography have beenused to provide such security; however, such schemes severely impactnetwork performance and are highly complex, thereby adding substantialcost and overhead.

Reliability is also critical to the wide adoption of mobile peer-to-peernetworks and is usually defined as the guarantee for data delivery inthe network. Due to the nature of the wireless medium, node and linkfailures are likely to occur. For example, nodes may need to communicateover harsh fading channels, thus enduring periods of intermittentfailures (resulting in packet losses) and even total loss ofconnectivity. Such a harsh environment does not provide adequateguarantees of reliability demanded for wider acceptance.

Two systems are disclosed herein which address the above-noted problemsinherent in conventional systems and methodologies; namely, aself-adapting scheme for robust and assisted mobile peer-to-peernetworks, and a multiplexing scheme for hybrid peer-to-peer networks. Itis to be appreciated, however, that the invention is not limited to thespecific systems and methods described herein. Rather, techniques of theinvention may be expanded and modified as will become apparent to thoseskilled in art given the teachings herein, and still remain within thescope of the present invention.

(1) Self-Adapting System for Robust and Assisted Mobile Peer-to-PeerNetworks

A system that aims at providing enhanced wireless communication securityand reliability of mobile peer-to-peer networks without compromising onefficiency. This system is adapted to address challenges in traditionalpeer-to-peer networks, such as the lack of centralized control andmonitoring, while not limiting their functionality or advantages, suchas ease and speed of deployment.

An exemplary system according to embodiments of the present inventionpreferably relies on one or more of the following components:

Assisted Mobile Peer-To-Peer Network (AMPN)—

is an architecture that integrates an out-of-band network (e.g., acellular network) to support and optimize the operation of thepeer-to-peer network. An agent, called Foreign Agent (FA), is deployedin the supporting network to monitor and collect information about thepeer-to-peer network by means of feedback mechanisms. The FA agentcommunicates with the mobile nodes in the peer-to-peer network via anout-of-band radio channel, thereby avoiding network disruption orcommunication overhead. The mobile nodes adapt their routing strategiesand behaviors based on feedback communicated by the FA agent to optimizesecurity and reliability. Note, that a primary role of the externalnetwork is to provide privileged access to the FA agent, but theexternal network could also be used to provide backbone access oroffload traffic.

AH-MNTP (Ad Hoc Multi-Network Protocol)—

an augmented version of MNTP that supports peer-to-peer communication.It is also designed to support communication with the FA agent andprovides for adaptive data transmission. AH-MNTP is a flexible protocol,allowing a mobile node to securely discover multiple node-disjoint pathsto a destination, eventually using these paths to maximize security,reliability and efficiency in the network. AH-MNTP allows a node toselectively adapt its routing strategy and data transmission behaviorbased, at least in part, on feedback from the FA agent. The defaultfallback position in case communication between a node and the FA isdisrupted is to maximize security. AH-MNTP preferably integrates amodified version of an Ad hoc On-demand Distance Vector (AODV) protocolfor route discovery and management. The modifications to AODV includethe support multi-path node-disjoint route discovery, secure routing aswell as mechanisms to support MTNP as the transport protocol.

FIG. 7 illustrates an exemplary assisted mobile peer-to-peer network700, according to an embodiment of the invention. Network 700 includes aplurality of individual cellular networks 702, 704, 706, each cellularnetwork supporting a plurality of corresponding mobile devices 708. Atleast a portion of a given cellular network may have a coverage areaassociated therewith (e.g., designated by dotted boundary lines) whichoverlaps at least a portion of one or more other cellular networkcoverage areas, as shown. At least a subset of the mobile devices 708may establish a per-to-peer network (e.g., ad hoc network), with eachmobile device forming a node in the peer-to-peer network. As apparentfrom FIG. 7, devices outside of a given cellular network may communicatewith one another as part of the peer-to-peer network. For example,mobile device 708 a in cellular network 702 and mobile device 708 b incellular network 704 may be part of the same peer-to-peer network.

As stated above, an FA agent 710 communicates with the mobile devices708 via one or more out-of-band radio channels established between atower 712 (base station) corresponding to a given cellular network 702and one or more mobile devices 708 in the cellular network. The FA agent710 is preferably configured as a watchdog agent deployed in theout-of-band network. Participating nodes in the peer-to-peer networkcommunicate with the FA agent 710 via one or more out-of-band channels714. The FA agent 710 is operative as a centralized server and maintainsglobal information about the entire peer-to-peer network.

In an illustrative embodiment, the FA agent 710 is configured forcollecting information, providing routing assistance and/or supportingsecurity in the peer-to-peer network. Collecting information mayinclude, for example, soliciting feedback from both sending andreceiving nodes (e.g., mobile devices 708) after successful orunsuccessful data transmission. Furthermore, or alternatively, suchinformation collection process may include compiling statistics or otherinformation, such as, for example, status on the reliability of nodes orthe lifetime of a particular route (where the lifetime of a given routemay be defined herein as an elapsed time from when a route wasoriginally established to the time it is first marked as failed).Collected statistics may be weighted over time to account for networkdynamics (e.g., network connectivity, node density, number of intruders,etc.). The FA agent 710 may provide route assistance, which may includemaintaining “blacklists” of nodes that must not be included in anycomputed routes. Furthermore, by providing feedback to the nodes (e.g.,mobile devices 708), the nodes will individually be able to assign trustlevels to route candidates upon completion of a route discovery process.The trust level assigned to individual nodes on a given route can beused to determine the overall trust level of the particular route. Thetrust level of a given node may be determined by its past performance,or other prescribed characteristics (e.g., length of time in service,etc.).

Security measures provided by the FA agent 710 preferably includeencrypted information exchanges with the mobile devices/nodes 708.Respective symmetric keys are exchanged as part of an authenticationprocess when a node is first joining the peer-to-peer network (which canrely on the built-in authentication provided by the out-of-band network,such as a cellular network). The FA agent may also be responsible forkey distribution and clock synchronization if required by the securityschemes used in the peer-to-peer network.

AH-MNTP is an augmented version of MNTP that integrates a modifiedversion of the AODV routing protocol for route discovery and management,called a Route Discovery (RD) module and a Route Analyzer (RA) module.The AODV protocol, as defined by RFC3561 (See, e.g., C. Perkins, et al.,“Request for Comments: 3561, Ad hoc On-Demand Distance Vector (AODV)Routing,” Network Working Group, July 2003, the disclosure of which isincorporated herein by reference in its entirety), defines routediscovery and data forwarding. According to embodiments of theinvention, the route discovery functionality is preferably modified asdescribed below to incorporate features of the invention.

Specifically, the RD module is adapted to perform a discovery ofavailable candidates to define a set of available paths between a sourcenode and a destination node. The set of paths discovered is then used bythe RA module which is operative to determine if they are suitable toMNTP. The discovery process is preferably modeled after AODVM to allownode-disjoint route discovery and determining nodal connectivity foreach route discovered, although alternative discovery techniques aresimilarly contemplated by the invention. Candidate paths, as well as amain path, are preferably tracked during route discovery, for example,by requiring intermediate nodes to construct a RREQ table, oralternative mechanism, for duplicate RREQ instead of discarding them asspecified in the AODV standard, according to embodiments of theinvention.

Knowledge of the actual nodal connectivity in the RD process allows apath-scheduling algorithm in AH-MNTP to optimize reliability andsecurity of the data transmission based, at least in part, on feedbackfrom the FA agent. Moreover, since no routing decision is left to thecorresponding nodes, the sender and receiver can explicitly correlatethe successful and/or failed transmission to the corresponding routes.Non-operational or potentially compromised routes are reported back tothe FA agent. Nodal connectivity is determined using a modified RREQ-ACKpacket to record the unique IDs of all intermediary nodes. In the AODVprotocol as defined by RFC3561, the RREQ-ACK only collects the uniqueIDs of the next hop. According to an aspect of the invention, RREQ-ACKis preferably modified such that all the unique IDs of all intermediarynodes along a given path are contained in the RREQ-ACK. Upon receptionof these packets, the sender is able to reconstruct the discovered pathand forward it to the FA agent for analysis.

The RA module correlates feedback from the FA agent with RD routeinformation in order to determine a set of optimal routes and amultiplexing data transmission strategy (e.g., dispersed, duplicated, ormultiplexed). The RA module accepts a route as valid if: (a) it isnode-disjoint with active routes; (b) none of the nodes involved areblacklisted; and (c) it has not been reported as failing within aprescribed time window. Routes can also be associated with a trustednode (e.g., as a function of past behavior) as recorded by the FA agent.

To determine an optimal multiplexing strategy for a given application,the RA module preferably always favors security and reliability overperformance, especially during a bootstrap phase; in that respect,packet duplication and dispersion are the most secure and reliablemultiplexing strategies. Once the FA agent can confirm thetrustworthiness of the certain paths, the RA module preferably controlsthe multiplexing to take advantage of the multiple paths to adestination in order to optimize performance.

When the interaction with the FA agent does not allow an assessment of aroute, the RA module will preferably assume that it is operating in anadverse environment. The feedback mechanism between the system and theFA agent preferably operates at two levels. Specifically, according toan embodiment of the invention, communication with the RD module isunidirectional; route updates are sent to the FA agent upon discovery.Communication between the RA module and the FA agent is preferablybidirectional, whereby the RA updates the FA agent upon transmission ofdata to a receiver. In other embodiments, a unidirectional communicationwith the RA module may be employed, although certain features of themethodology may be unavailable in this case. A successful transmissionwill preferably increase the rating and trustworthiness of all the nodesin the route while a failed transmission will decrease the rating forthe route and all intermediate nodes in at least the failed portion ofthe route. Based on data collected by all participating nodes, the FAagent is able to continuously (i.e., dynamically) refine the quality andtrustworthiness of all participating nodes in the peer-to-peer network.

(2) Mobile Hybrid Ad Hoc Network Multiplexing

A system that provides data path aggregation in mobile hybridpeer-to-peer networks will be described in further detail herein below.Mobile hybrid peer-to-peer networks may be defined herein as a type ofnetwork comprised of multi-homed devices that support both peer-to-peercommunications and infrastructure-based communications concurrently. Asimple example of such a device is a smartphone equipped with Wi-Fiand/or Bluetooth for peer-to-peer communication and a 3G interface forinfrastructure-based communication. Of course, one could easily conceiveof a device with many more network interfaces, such as a Bluetoothand/or UWB used for peer-to-peer communication and Wi-Fi, 3G and/or 4Gcellular used for infrastructure-based communication. In such a system,nodes can communicate directly with each other over the mobilepeer-to-peer network and with external infrastructure networks.

Nodes can also preferably advertise their infrastructure networkcapabilities to each other. Infrastructure networks capabilitiesinformation may include, for example, network interface type, gatewayand routing information, QoS information, pricing, location information,in addition to local information such as current load, networkutilization, battery level, among other information. This mechanismallows extending the communication and routing of traffic outside themobile peer-to-peer network.

FIG. 8 illustrates an exemplary four-node hybrid peer-to-peer network800, according to an embodiment of the invention. As shown in thefigure, network 800 includes a first node (A) 802, a second node (B)804, a third node (C) 806 and a fourth node (D) 808. Each of the nodescomprises a mobile device.

After a discovery and mutual authentication phase, nodes A, B, C and Dform a mobile peer-to-peer network and preferably exchange route updatesto reflect their mobility. They may also advertise network capabilitiesinformation, allowing information to be routed into and out of the localpeer-to-peer network. For instance, with reference to FIG. 8, nodes Aand B are out of reach of any infrastructure network, while nodes C andD have advertised their ability to attach to a Wi-Fi network and a 3Gcellular network for node C and a 3G cellular network for node D.

When routing traffic bound outside of the local peer-to-peer network,node A for instance, has the option to use node C or node D as a gatewayto outside networks. If node C is selected, data traffic 810 will beforwarded to node B, which will then forward said traffic to node C onbehalf of node A. Node C will then route the traffic 810 to the Wi-Finetwork or the 3G cellular network based on its advertised networkcapabilities information. Alternatively, node A could have chosen node Das the final gateway, in which, node C would have acted as a forwarder(as in the case of node B) and not as a gateway. The sender (node A) canchoose the gateway based on a number of factors, including routefreshness (exchanged during periodic route updates between peer-to-peernodes), signal quality and network utilization (for the infrastructurenetwork interface), policies (e.g., routing preferences, price, securityconsideration, etc.), among other factors.

Furthermore, since each node 802, 804, 806, 808 forming the mobilepeer-to-peer network has multi-path aggregation capabilities, they canaggregate remote connections in addition to local connections. Remoteconnections may be defined herein as network connections attached topeer nodes, reachable over a peer-to-peer network. A sender node canrequest that its traffic be routed over multiple remote connections in aload-balancing mode (e.g., connections are not split but mapped to aspecific remote interface(s)) or in an aggregated mode (e.g.,connections are split and transmitted concurrently over multiple remoteinterfaces).

The aggregated mode preferably requires an aggregation server, oralternative controller, located outside of the peer-to-peer network andreachable by the nodes used for the aggregation through their remoteconnections. Sending and receiving nodes will preferably advertise theirextended multi-homed capabilities to the server, which include theirlocal network interfaces and the remote interfaces of peer nodes inreach. Periodic updates to the server are made as peer-to-peer nodesreceive route updates from other peer nodes in the peer-to-peer network.This mechanism allows nodes to use different local and remoteconnections for uplink and downlink, but more importantly minimizes lossand retransmissions as peer-to-peer routes are breaking or expiring, acommon and expected behavior of mobile (wireless) peer-to-peer networks

The aggregation scheduling strategy can be computed by the sender basedat least in part on prescribed (e.g., agreed upon a priori) QoSguarantees and traffic priority, policies (e.g., predefined ordisseminated by the aggregation server), or a best effort servicewhereby peer nodes apply a store-and-forward strategy, transmitting aportion of the traffic (e.g., based on current or expected network load,power level, etc.) and forwarding the remainder of the traffic to otherparticipating nodes.

With reference to FIGS. 9A and 9B, consider the following illustrativescenario whereby nodes A, B and C have one remote connection and node Dhas two remote connections. Node A initiates a download of a filelocated on an Internet server. Route discovery has been achieved andperiodic route updates are sent between peer nodes. An aggregationserver 912, and corresponding infrastructure network 914 incommunication with the aggregation server, preferably receives theperiodic route updates and other status information relating to thenodes for managing the peer-to-peer network. Nodes B and C have agreedto route node A's traffic on their remote interfaces, while node D hasagreed to route traffic on only one of its remote interfaces, with theother remote interface being reserved for high-priority traffic. Node Ais able to download the file through an aggregation of its own localconnection 952, and three additional remote connections 954, 956 and958, from nodes B, C and D, respectively, to thereby increase bandwidth,as shown in FIG. 9B. Once the download of the file is finished, if ahigh-priority file needs to be transmitted, the second remote connection960 associated with node D is seamlessly added to the aggregation.

With reference now to FIGS. 10A and 10B, consider an illustrativescenario similar to the one described above in conjunction with FIGS. 9Aand 9B, but during the download, node A receives an update from node Bthat node C is no longer reachable from node B (i.e., the connectionbetween nodes B and C has failed). Node A immediately emits an update tothe aggregation server 912 that it is no longer reachable through nodesC and D. Similarly, upon detecting the link break, node C transmits anupdate to the aggregation server 912 that its remote connection is nolonger valid for node A. Upon receiving the first of the updates, theaggregation server 912 immediately brokers a relocation of connectionendpoints (e.g., achieved through socket migration) from nodes C and Dto nodes A and B, allowing the in-flight traffic to be delivered to nodeA, thereby minimizing the amount of data in need of retransmission.

The relocation of connection endpoints may be subject to predeterminedpolicies residing on the server 912 or through a negotiation processwith each of the participating nodes, brokered by the server, or throughsome other mechanism. By default, connection endpoints are relocated tonode A if its local connection is reachable by the server, or otherwiseto the closest node (e.g., “closeness” being defined by number of hops)to node A; node B in this example.

An exemplary applicability of the system to real-world scenarios aredescribed below, according to illustrative embodiments of the invention.

Scenario #1:

Assume a disaster-response scenario where first responders that are partof a recovery mission are equipped with Wi-Fi enabled PC tablets forsituational analysis as well 3G cellular and Wi-Fi capable handsetdevices, in addition to a few strategically deployed command and controlvehicles equipped with high-speed wireless connection such as satelliteor microwave. Furthermore, during the disaster all infrastructure hasbeen destroyed in a 10-miles radius blackout zone. As first respondersperform their duties, they are able to communicate with each other usingthe peer-to-peer network established using the Wi-Fi network interface.One could imagine a team leader disseminating orders, audio and/or videofiles from a command and control vehicle through the peer-to-peernetwork and aggregating the satellite and 3G cellular connections offirst responders in the coverage zone to increase the availablebandwidth.

Similarly, data traffic from first responders inside the blackout zoneis relayed through peers and routed through their respective remoteconnections, if available, or through the satellite connection of thecommand and control vehicle. As service providers are restoring servicein part of the blackout zone, peer devices are detecting and advertising(updating) their new capabilities, allowing resources to be expandeddynamically without user intervention.

To ensure proper segregation of information, one could envision a policywhere all information from team members is funneled through the teamleader's system (hierarchical behavior). The default policy becomes: allcommunications from first responders is propagated to the command andcontrol vehicle using the peer-to-peer network which in turn aggregatesthe active remove connections on teammates' devices, maximizing theavailable bandwidth. In such a scenario, the supporting server wouldideally be deployed at HQ or central command.

Another application in this scenario would be to aggregate multiplenode-disjoint routes to retrieve shared resources available on peerdevices. For instance, a first responder could have large maps stored onone of the PC tablets and another first responder at the other end ofthe peer-to-peer network could retrieve it locally, aggregating multipleroutes available between him and his peer where the maps are stored,ensuring the best performance.

Scenario #2:

Assume a scenario where a group of Wi-Fi and 3G capable smartphone usersare at a venue (e.g., concert, cafe, shopping center, etc.). On average,only a portion of the users are using their respective Internetconnections at the same time. Users can form a peer-to-peer network anduse the 3G connection of other users with dormant radios to increase thenetwork performance of an application(s) running on their devices.Similarly, should a service operators' base-station be overloaded orunreachable, or coverage be nonexistent at a specific location, users ofthat service would be able to remain connected by using the connectionof nearby users, in a completely transparent fashion.

Scenario #3:

Consider a scenario taking place on a highway with cars equipped with 3Gcellular and Wi-Fi capable terminals. Cars are forming a vehicularpeer-to-peer network using the Wi-Fi interface and maintain connectivitywith each other dynamically; peer cars are added and removed from thepeer-to-peer network as they travel to their respective destinations.Assume that all the cars are using an Internet-based navigation systemthat retrieves high definition mapping information in real-time. Insteadof having all cars download the entire map for their location, maptitles already downloaded by peer cars could be exchanged rapidlybetween all the cars in the vicinity using one or more routes availablein the peer-to-peer network, thereby greatly improving performance andreducing the load on the 3G cellular network.

A practical implementation of this system could be achieved by deployinga TPA (as described above) augmented with AH-MNTP capabilities(described above) on each of the peer devices. The TPP protocol could beused for out-of-band signaling. Resource advertisements could beimplemented with an XMPP-based layer, using multicast for nodesbelonging to a peer-to-peer network.

As will be appreciated by those skilled in the art, aspects of thepresent invention may be embodied as a system, method or computerprogram product. Accordingly, aspects of the present invention may takethe form of an entirely hardware embodiment, an entirely softwareembodiment (including firmware, resident software, micro-code, etc.) oran embodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

One or more embodiments of the invention, or elements thereof, can beimplemented in the form of an apparatus including a memory and at leastone processor that is coupled to the memory and operative to performexemplary method steps.

One or more embodiments can make use of software running on a generalpurpose computer, workstation or alternative electronic system. Withreference to FIG. 11, such an implementation might employ, for example,a processor 1120, a memory 1130, and an input/output interface 1140formed, for example, by a display and a keyboard (not explicitly shown),or other peripheral components. The term “processor” as used herein isintended to include any processing device, such as, for example, onethat includes a CPU (central processing unit) and/or other forms ofprocessing circuitry. Further, the term “processor” may refer to morethan one individual processor. The term “memory” is intended to includememory associated with a processor or CPU, such as, for example, RAM(random access memory), ROM (read only memory), a fixed memory device(for example, hard drive), a removable memory device (for example,diskette), a flash memory and the like. In addition, the phrase“input/output interface” as used herein, is intended to include, forexample, one or more mechanisms for inputting data to the processingunit (for example, mouse), and one or more mechanisms for providingresults associated with the processing unit (for example, display). Theprocessor 1120, memory 1130, and input/output interface can beinterconnected, for example, via a bus as part of a data processing unit1100. Suitable interconnections, for example via network connection1150, can also be provided to a network interface, such as a networkcard, which can be provided to interface with a computer network, and toa media interface, such as a diskette or CD-ROM drive, which can beprovided to interface with media.

Accordingly, computer software including instructions or code forperforming the methodologies (processes) 1180 of the invention, asdescribed herein, may be stored in one or more of the associated memorydevices 1130 (for example, ROM, fixed or removable memory) and, whenready to be utilized, loaded in part or in whole (for example, into RAM)and implemented by a CPU. Such software could include, but is notlimited to, firmware, resident software, microcode, and the like.

A data processing system suitable for storing and/or executing programcode will include at least one processor 1120 coupled directly orindirectly to memory elements 1130 through a system bus or alternativeconnection means. The memory elements can include local memory employedduring actual implementation of the program code, bulk storage, andcache memories which provide temporary storage of at least some programcode in order to reduce the number of times code must be retrieved frombulk storage during implementation.

Input/output or I/O devices 1140 (including but not limited tokeyboards, displays, pointing devices, and the like) can be coupled tothe system either directly (such as via a bus) or through interveningI/O controllers (omitted for clarity).

Network adapters such as a network interface may also be coupled to thesystem to enable the data processing system to become coupled to otherdata processing systems or remote printers or storage devices throughintervening private or public networks. Modems, cable modem and Ethernetcards are just a few of the currently available types of networkadapters.

As used herein, including the claims, a “server” includes a physicaldata processing system (for example, system 1100 as shown in FIG. 11)running a server program. It will be understood that such a physicalserver may or may not include a display and keyboard.

As noted, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon. Anycombination of one or more computer readable medium(s) may be utilized.The computer readable medium may be a computer readable signal medium ora computer readable storage medium. A computer readable storage mediummay be, for example, but not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,or device, or any suitable combination of the foregoing. More specificexamples (a non-exhaustive list) of the computer readable storage mediumwould include the following: an electrical connection having one or morewires, a portable computer diskette, a hard disk, a random access memory(RAM), a read-only memory (ROM), an erasable programmable read-onlymemory (EPROM or Flash memory), an optical fiber, a portable compactdisc read-only memory (CD-ROM), an optical storage device, a magneticstorage device, or any suitable combination of the foregoing. In thecontext of this document, a computer readable storage medium may be anytangible medium that can contain, or store a program for use by or inconnection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++, or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described herein with reference toillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products according to embodiments of the invention. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the illustrations and/orblock diagrams, may be implemented by computer program instructions.These computer program instructions may be provided to a processor of ageneral purpose computer, special purpose computer, or otherprogrammable data processing apparatus to produce a machine, such thatthe instructions, which execute via the processor of the computer orother programmable data processing apparatus, create means forimplementing the functions/acts specified in the illustrations and/orblock diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus, or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the illustrations and/orblock diagram block or blocks described herein.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theillustrations or block diagrams may represent a module, segment, orportion of code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should be noted thateach block of the block diagrams and/or illustrations, and combinationsof blocks in the block diagrams and/or illustrations, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

It should be further noted that any of the methods described herein caninclude an additional step of providing a system comprising distinctsoftware modules embodied on a computer readable storage medium; themodules can include, for example, any or all of the elements depicted inthe block diagrams and/or described herein; by way of example and notlimitation, a data acquisition module, an analysis module, and a dataprocessing module. The method steps can then be carried out using thedistinct software modules and/or sub-modules of the system, as describedabove, executing on one or more hardware processors 1120. Further, acomputer program product can include a computer-readable storage mediumwith code adapted to be implemented to carry out one or more methodsteps described herein, including the provision of the system with thedistinct software modules.

In any case, it should be understood that the components illustratedherein may be implemented in various forms of hardware, software, orcombinations thereof; for example, application specific integratedcircuit(s) (ASICS), functional circuitry, one or more appropriatelyprogrammed general purpose digital computers with associated memory, andthe like. Given the teachings of the invention provided herein, one ofordinary skill in the related art will be able to contemplate otherimplementations of the components of the invention.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

Although illustrative embodiments of the present invention have beendescribed herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various other changes and modifications may bemade therein by one skilled in the art without departing from the scopeof the appended claims.

What is claimed is:
 1. A method for forming an optimized communicationconnection providing enhanced performance of at least one applicationutilizing the communication connection, the optimized communicationconnection comprising a plurality of individual communication networks,the method comprising: obtaining a set of performance requirementscorresponding to the at least one application utilizing thecommunication connection; obtaining capacity information for each of aplurality of available channels associated with the respective pluralityof individual communication networks; applying at least one policy-basedmanagement criteria to the available channels for controlling one ormore aspects of the available channels; dynamically aggregating theplurality of individual communication networks to form the optimizedcommunication connection, the communication connection leveraging one ormore features and capabilities of at least a subset of the plurality ofcommunication networks; and controlling real-time traffic schedulingacross at least a subset of the plurality of available channels so as toadapt the communication connection to changes in at least one of networkconditions and policy-based management criteria.
 2. The method of claim1, wherein the step of obtaining a set of performance requirementscorresponding to the at least one application comprises performingreal-time profiling of the at least one application.
 3. The method ofclaim 1, wherein the step of applying at least one policy-basedmanagement criteria to the available channels is controlled locally. 4.The method of claim 1, wherein the step of applying at least onepolicy-based management criteria to the available channels is networkcontrolled.
 5. The method of claim 1, wherein the step of obtainingcapacity information for each of the plurality of available channelscomprises performing at least one of passive monitoring, activemonitoring and real-time data collection.
 6. The method of claim 5,wherein passive monitoring comprises monitoring traffic performance onthe plurality of available channels in real-time.
 7. The method of claim5, wherein active monitoring comprises periodically sending one or moreactive performance probes.
 8. The method of claim 1, wherein the step ofcontrolling real-time traffic scheduling across the plurality ofavailable channels comprises estimating at least one performancecharacteristic of each of the plurality of channels to thereby optimizeconcurrent transmission of data traffic.
 9. The method of claim 8,wherein the at least one performance characteristic comprises at leastone of bandwidth, round trip delay time (RTT) and jitter associated witheach of the plurality of channels.
 10. The method of claim 1, whereinthe step of dynamically aggregating the plurality of individualcommunication networks to form the optimized communication connection isperformed using a dynamic multi-path network protocol.
 11. The method ofclaim 1, wherein the step of controlling real-time traffic schedulingcomprises reconfiguring real-time traffic scheduling across at least thesubset of the plurality of available channels as a function of at leastone of available window size, latency estimation, bandwidth estimationand in-flight and queue traffic computation corresponding to at leastthe subset of the plurality of available channels.
 12. The method ofclaim 1, wherein the step of controlling real-time traffic schedulingcomprises reconfiguring real-time traffic scheduling across at least thesubset of the plurality of available channels as a function of one ormore requirements of the at least one application utilizing thecommunication connection.
 13. The method of claim 12, wherein the stepof reconfiguring real-time traffic scheduling comprises at least one ofmirroring traffic on a subset of the plurality of available channels,and mapping at least a portion of prescribed traffic onto at least onespecified network path.
 14. The method of claim 1, further comprisingthe step of maintaining at least one virtual queue per destination witha sender's retransmission queue and applying a retransmission procedureon a per-destination basis without affecting an operation of the senderso as to enable the sender to adapt a size of a congestion windowmaintained by the sender to thereby reduce a number of unnecessarypacket retransmissions when packet reordering is performed at thedestination.
 15. The method of claim 1, further comprising: obtainingtransmission destination information for determining in-order deliveryof data by a sender to a corresponding destination; and generatingselective acknowledgments to the sender indicative of receipt of thedata by the destination as a function of the transmission destinationinformation.
 16. The method of claim 15, wherein the step of generatingselective acknowledgments comprises delaying all acknowledgments to thesender irrespective of an order in which data is received.
 17. Themethod of claim 1, further comprising establishing a plurality oftransport streams within the optimized communication connection, each ofat least a subset of the streams being configured to have differentperformance characteristics.
 18. The method of claim 17, wherein thestep of controlling real-time traffic scheduling comprises applying adifferent scheduling scheme to each of at least two of said plurality oftransport streams.
 19. The method of claim 1, wherein the step ofapplying at least one policy-based management criteria to the availablechannels for controlling one or more aspects of the available channelscomprises controlling one or more aspects of the available channels inreal-time.
 20. A method for performing managed data offloading usingmultiple access networks, the method comprising: provisioning a clientdevice to establish simultaneous connectivity with the client deviceusing at least two channels in the multiple access networks, such thatdata from a first one of the channels is offloaded to a second one ofthe channels in real-time without disruption of service; adding a layerof signaling between the client device and at least one server device incommunication with the client device; dynamically aggregating at least asubset of available access networks as a function of decision-makingcriteria controlled jointly by a transaction between the client andserver devices; and controlling offloading of at least a portion of dataas a function of one or more characteristics of the available accessnetworks.
 21. An apparatus for multiplexing a plurality of individualnetwork connections to form an enhanced communication connectionproviding at least one of increased bandwidth, security, reliability andefficiency in a hybrid peer-to-peer network, the apparatus comprising:memory; at least one processor connected with the memory; and anon-transient persistent storage medium that contains instructionswhich, when loaded into said memory, configure said at least oneprocessor: (i) to discover and maintain multiple-network connectivityfor at least a subset of a plurality of respective peers forming apeer-to-peer ad hoc network; (ii) to leverage a plurality of individualmultiple access networks connected with one or more of the plurality ofpeers and aggregate the plurality of multiple access networks into alogical connection; and (iii) to implement out-of-band signalingoperative to support communication and control of the peer-to-peer adhoc network.
 22. The apparatus of claim 21, wherein the at least oneprocessor is operative to aggregate the plurality of multiple accessnetworks into the logical connection such that the logical connectionintegrates at least a portion of characteristics associated with theplurality of individual multiple access networks, whereby the logicalconnection has overall characteristics associated therewith that aresuperior to any one of the individual multiple access networks.